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Executive  Summary 

Proposition  E - The  Muni  Reform  Initiative 

On  November  2,  1999,  the  voters  of  San  Francisco  overwhelmingly 
approved  Proposition  E,  the  most  substantial  reformation  in  Muni 
history.  The  voters’  intent  was  to  institute  structural,  administrative 
and  financial  reforms  designed  to  provide  Muni  with  the  "resources, 
independence  and  focus  necessary"  to  become  one  of  the  best  urban 
transit  systems  in  the  world.  Recognizing  the  City’s  dependence  on 
public  transit  and  its  need  for  efficient  and  reliable  transit  service 
that  can  compete  with  the  private  automobile,  the  drafters  of  the 
initiative  sought  to  restructure  the  City's  provision  and  administration 
of  transportation  and  parking  services  and  strengthen  the  City's 
Transit-First  Policy. 

The  overall  goals  for  transit  service  articulated  in  Proposition  E 
(now  Article  VINA  of  the  San  Francisco  City  Charter)  are  as  follows 
(Section  8A.  1 00): 

1.  Reliable,  safe,  timely,  frequent,  and  convenient  service  to  all 
neighborhoods: 

2.  A reduction  in  breakdowns,  delays, over-crowding,  preventable 
accidents: 

3.  Clean  and  comfortable  vehicles  and  stations,  operated  by 
competent,  courteous,  and  well-trained  employees: 

4.  Support  and  accommodation  of  the  special  transportation 
needs  of  the  elderly  and  the  disabled: 

5.  Protection  from  crime  and  inappropriate  passenger  behavior 
on  the  Municipal  Railway:  and 

6.  Responsive,  efficient,  and  accountable  management. 
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To  achieve  these  goals,  Article  VIllA  created  the  Municipal 
Transportation  Agency  (MTA),  combining  the  responsibility  for  street 
operation  (Department  of  Parking  and  Traffic)  with  the  dominant 
"user"  of  the  streets  - Muni.  Article  VIllA  also  established  service 
standards  and  accountability  measures,  and  required  an  independent, 
biennial  quality  review  of  transit  operations.  This  report  represents 
the  findings  of  the  first  independent  review  of  Muni’s  performance 
since  the  passage  of  Proposition  E. 
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Proposition  E Service  Standards 

The  service  standards  (or  performance  measures)  adopted  under 
Proposition  E were  not  intended  to  create  onerous  reporting 
requirements,  but  rather  to  provide  Muni  with  the  tools  needed  to 
create  a world-class  transit  service.  In  order  to  do  this  effectively, 
the  service  standards  need  to  provide  information  and  feedback 
that  Muni’s  management  can  readily  use  to  help  shape  decisions  and 
policies  so  that  the  desired  outcomes  can  bo  achieved. 

While  Proposition  E specifically  stated  the  method  of  measurement 
and  goals  for  several  of  the  service  standards,  it  also  provided  some 
flexibility  with  regard  to  the  way  in  which  other  standards  could  be 
measured  and  the  milestones  or  goals  to  be  achieved.  When  not 
specified  by  Proposition  E.  the  MTA  Board  adopted  methods  and 
definitions  of  measurement  as  well  as  specific  goals  and  milestones 
for  cacli  of  the  service  standards.  Additionally,  Section  8A.  104  of 
the  City  Charter  allows  the  MTA  Board  to  vote  to  amend  any  of 
the  service  standards  (after  holding  a public  hearing  on  any  such 
amendments). 

Muni's  Citizen's  Ativisory  Committee  (CAC)  and  the  MTA  Board 
review  Muni's  performance  quarterly,  and  review  the  definitions  of 
measurement,  methods  of  measurement  and  the  goals  for  each  of 
the  service  standards  annually.  Muni  publishes  quarterly  Service 
Standards  Reports  which  include  a description  of  each  of  the  Service 
Standards  and  a summary  of  Muni's  performance.  (These  reports 
arc  available  to  the  public  via  Muni’s  web  site.)  These  reports  also 
inclutic  atiditionai  performance  information  that  is  not  required 
by  Proposition  E.  but  is  used  by  Muni  for  other  pui'poses,  such  as 
employee  incentive  programs. 


Table  ES-I  summarizes  the  service  standards  that  are  required  by 
Proposition  E,  the  actual  measurements  that  are  made,  the  goals 
or  milestones  that  have  been  established,  and  overall  performance 
during  the  period  covered  by  this  review  (FY2001  and  FY2002).  (For 
a more  detailed  discussion  of  performance,  please  see  the  body  of 
this  report.) 
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Table  ES- 1 : Proposition  E Service  Standards  and  Performance 


1 Service  Stenderd 

Prop  E Requirements 

Measurement 

Comminti 

Goal  or  Miloitono 

Svitemwido  Porformonca 
(Audited  for  FY2001  and 

FV9n09^  linftiiriltari  fnr  CV9nO^\ 

A.  SYSTEM  RELIABILITY 

la.  % of  vehicles  that  run 
on  time  according  to 
published  schedules. 

On-time  performance 
standard:85%  of 
vehicles  must  be  on-time 
(no  more  than  1 minute 
early  or  four  minutes 
late  as  measured  against 
a published  schedule). 

Actual  arrival  times 
compared  against  published 
times  (quarterly  average 
for  systemwide,  mode,  and 
division). 

Specifically  measures 
what  is  required  by  Prop 
E. 

FY2001:  65% 
FY2002:  70% 
FY2003:  75% 
FY2000:  85% 

FY2001:  goal  nchiovod 
FY2002;  goal  acliinvod 
FY2003;  fnilod  to  iicliiovu  goal 

2a.  % of  scheduled  service 
hours  that  are  delivered 
and  % of  scheduled 
vehicles  that  begin 
service  at  the  scheduled 
time. 

Service  delivery 
standard:  98.5%  of 
scheduled  service  hours 
must  be  delivered, 
and98.5%  of  scheduled 
vehicles  must  begin 
service  at  the  scheduled 
time. 

- % operator  availability 
(quarterly  for  systemwide, 
mode,  and  division) 

- % equipment  availability 
(quarterly  for  systemwide, 
mode,  and  division) 

-%  total  availability 
(quarterly  for  systemwide, 
mode,  and  division) 

- % of  late  pull-outs 
(quarterly  for  systemwide, 
mode,  mode  and  division) 

Specifically  measures 
what  is  required  by 
Prop  E;  however  Prop  E 
does  not  state  that  % 
of  hours  delivered  for 
operator  availability  and 
equipment  availability  bo 
specified  separately. 

FY200I:  96.5% 
FY2002:  97% 
FY2003:  97.5% 
FY2004:  98.5% 

FY20Q1;  failed  to  achiovogoiil 
FY2002:  goal  nchiovod 
FY2003:  failed  to  achiovo  goal 

3a.  Missed  service  due 
to  either  insufficient 
vehicles  or  driver 
unavailability  as 
a percentage  of 
scheduled  service 
hours. 

Missed  service  due 
to  either  insufficient 
vehicles  or  driver 
unavailability  as  a 
percentage  of  scheduled 
service  hours. 

- % hrs  missed  due  to 
operator  availability 
(quarterly  for  systemwide, 
mode,  and  division) 

- % hrs  missed  due  to 
equipment  availability 
(quarterly  for  systemwide, 
mode,  and  division) 

- % of  total  hrs  missed 
(quarterly  for  systemwide, 

i 4 

Specifically  measures 
what  is  required  by 
Prop  E;  however  Prop 
E does  not  state  that 
S of  hours  missed 
due  to  operator 
unavailability  and 
equipment  unavailability 
be  specified  separately. 
This  standard  is  simply 

. i 

FY2001:  3.5% 
FY2002:  3% 
FY2003:  2.5% 
FY2004:  1.5% 

FY2001:  failed  to  achiovo  goal 
FY2002:  goal  achiovod 
FY2003:  failed  to  achiovo  goal 

Service  standard  Prop  E Roquiromo^ 


I /la.  % of  vohinlcs  that  pass 
publishmi  timo  points 
ilminii  moasiiromont 
poriods  iiiifiblo  to  pick 
up  passonpors  dim 
to  crowdini)  witbnnt 
buini)  lollowod  witbiii 
3 minutfls  or  loss  by 
imotbor  vnhidn  on 
tho  samo  rotilo  with 
spoco  fur  ail  waitiim 
|)nssoiiiiors. 

5a.  Poak  potiod  passnupor 
load  factors. 


I 6n.  Actual  boadways 
npainst  scbndulod 
boadways  on  all  radial, 
oxpioss,  cross  tuwn, 
socoiulary  and  fuodnt 
linos  for  poak.  bnso. 
nvoninp  and  Into  nipltt 
sorvicos, 

7n.  % vobicln  availnbility 
and  rnliability  (moan 
distnnco  bnlwoun 
lailmol  by  imnlo. 


% of  vGhiclos  that  pass 
publishod  timo  points 
durinp  moasuromont 
poriods  unablo  to  pick 
up  passonpors  duo 
to  crowdinp  without 
boinp  followtal  within 
3 minutes  or  loss  by 
anothoT  vohiclo  on 
Ihn  samo  routo  with 
spncQ  lor  all  waitiiip 
passonpors. 

Poak  poriod  passonpnr 
I load  factors. 


Measurement 


% of  vehicles  passing 
I by  passengers  due 
I to  overcrowded 
I vehiclolriuarterly  for 
systemwide,  mode,  and 

I division). 


Actual  boadways  against 
scbfldulod  boadways  on 
all  radial,  oxpross.  cross 
I town,  sucDiulary  ami 
loodor  linos  lor  peak, 

I bnso,  ovoning  and  Into 
night  sorvicos. 

% vohiclo  nvniinbility  and 
rolinhility  {monn  distnnco 
I hotwoon  fniliiro)  by 
I mode. 


% ol  passonpors  onboard 
vohiclo  that  oxcood 
combinod  seating  and 

standing  cnpacityiquartQrly 

for  systomwido,  mode,  and 
division). 

% rovomio  vehicles  adhering 
to  schodulod  headways 
(t]unrtorly  tor  systemwide, 
mode,  and  division). 


% vohicle  availability 
(quarterly  for  systemwide, 
I modo.  and  division). 


Comments 


ITA  Board  adopted  a 
sampling  methodology 
to  determine  percent  of 
jass  ups  systemwide 
and  for  each  mode. 

MTA  Board  also  adopted 
a specific  goal. 


MTA  Board  adopted  a 
definition  and  method 
of  measurement  and  a 
specific  goal. 


MTA  Board  adopted  a 
definition  and  method 
of  measurement  and 
specific  goals. 


Goal  or  Milestone 


Systemwide  Performance 
(Audited  for  FY20Q1  and 
FY2002;  Unaudited  for  FY2003) 


I FY2001:  goal  achieved 
FY2002;  goal  achieved 
FY2003:  failed  to  achieve  goal  for 
four  out  of  12  months 


^ 85%  of  combined 
seating  & standing 
capacity 


FY2001:  S80% 

I FY2002,  FY2003, 
FY3004:  ^ 85%  of 
, the  time  within  30%  or 
10  minutes  of  scheduled 
I headway  (whichever  is 
less) 


Only  report  % vehicle 
availability.  Reliability  is 
reported  under  Service 
Standard  9a.  MTA 
Board  adopted  a specific 
goal. 


; 98.5%  vehicle 
availability 


FY2001:  failed  to  achieve  goal 
FY2002:  failed  to  achieve  goal 
FY2003:  failed  to  achieve  goal 


FY2001;  failed  to  achieve  goal 
FY2002:  failed  to  achieve  goal 
FY2003:  failed  to  achieve  goal 


FY2001:  goal  achieved  (except 
j for  light  rail) 

i FY2002;  goal  achieved  (except 
for  light  rail) 

FY2003:  goal  achieved  (except 
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Service  Standard 

Prop  E Requirements 

Measurement 

Comments 

Goil  or  Milostone 

Svstemwida  Performonca  I 

(Audited  for  FY2QQ1  and  1 

FY2002;  Unaudited  for  FY2Q03)  1 

8a.  Unscheduled  absences 
by  operator,  mechanical 
and  administrative 
personnel. 

Unscheduled  absences 
by  operator,  mechanical 
and  administrative 
personnel. 

% of  absences  not 
scheduled  in  advance  by 
personnel  (quarterly). 

MTA  Board  adopted  a 
definition  and  method 
of  measuromont  and 
specific  goals.  Also 
usQ  data  for  omployoo 
incentivo  goals. 

Annual  G%  reduction  of 
YTO  for  Maintenance 
(Mechanical), 

10%  reduction  for 
Transportation 
(Operators),  and  5% 
reduction  for  Admin 

FY2Q01:  failed  to  achieve  goal  for 
all  divisions 

FY2002:  Administration  Division 
failed  to  achieve  goal;  goal 
achieved  tor  othor  divisions 
FY2003:  goal  achieved  for  all 
divisions 

9a.  Miles  between  roadcalls 
btty  mode. 

Miles  between  roadcalls 
by  mode. 

Miles  between  roadcalls 
or  mean  distance  between 
failures  (MDBF)  (quarterly 
for  systemwide,  mode,  and 
division). 

MTA  Board  adopted  a 
definition  and  mothod 
of  moasurQiiiont  and 
specific  goals. 

Increase  miles  hotwoon 
roadcalls  - specific  gonls 
are  ostulilishod  each 
fiscal  year  and  iiuartur 
for  each  division 

FY2001;  seven  out  of  ten 
divisions  achieved  goal 
FY2002;  five  out  ef  nine  divisions 
achiovod  goal 

FY2003;  6 out  of  nine  divisions 
achieved  goni 

|B.  SYSTEM  PERFORMAiyCE  | 

lb.  Passengers  carried  by 
mode. 

Passengers  carried  by 
mode. 

- It  o\  people  boarding 
revenue  vehicles  (annually 
for  systemwide  and  model. 

- % change  in  ridership  from 
previous  year  (annually  for 
systemwide  and  mode). 

MTA  Board  adopted  a 
dofinition  and  mothod 
of  measuromont  and 
specific  goals. 

FY2001  andFY2002: 
2%  Increaso  over 
previous  year 
FY2003  nndfY?00n: 
224  M passengers 

FY2001:  goal  achieved 
FY2002:  goal  achieved 
FY2003:  failed  to  achieve  goal 

2b,  Fare  revenues 
generated  by  mode. 

Fare  revenues  generated 
by  mode. 

- Onboard  fare  collected  on 
revenue  vehicles  (annually)- 
Sales  of  individual  tickets 
(annually) 

- Sales  of  passes  (annually) 

- Muni's  share  of  regional 
passes  sold  (annually) 

MTA  Board  adopted  a 
definition  and  method 
of  measurement  and 
specific  goals. 

FY200I  aridFY2002: 
increase  faro  revenue  by 
$1.6M  over  previous  year 
FY2003:  $100  Min  faro 
revenue 

FY2004:  $117  Min  faro  ' 
revenue 

FY2001:  goal  achievral 
FY2002:  goal  acldevod 
FY2003:  failed  to  achiove  goal 
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1 Service  Stenderd 

Prop  E Requirements 

Measurement 

Comments 

Goal  or  Milestone 

Systemwide  Performance 
(Audited  for  FY2001  and 
FY2002;  Unaudited  for  FY2003) 

3h.  Hours  and  milos 
oporatfid  hy  riiodn. 

Hours  and  milos 
oporatod  hy  modo. 

- fl  of  vehicles'  revenue 
hours  (annually  by  mode) 
-//  of  vehicles'  revenue 
milos  (annually  hy  mode) 

MTA  Board  adopted  a 
definition  and  method 
of  measurement  and 
specific  goals. 

FY2001  and  FY2002: 
1.2%  increase  in 
revenue  miles  and 
revenue  hoursFY2003: 
3.6M  hours  and  25M 
milesFY2004:  3.5M 
hours  and  25M  miles 

FY2001:  goal  achieved 
FY2002;  goal  achieved 
FY2003:  goal  achieved  for  miles; 
failed  to  achieve  goal  for  hours 

dll.  Expnnsos  inciirrod  hy 
niodo. 

Exponsos  inciirrod  hy 
modo. 

Only  reports  whether  or  not 
costs  wore  within  hiidoot. 
Does  not  report  actual 
expenses  incurred  hy  modo. 

MTA  Board  adopted  a 
definition  and  method 
of  measurement  and 
specific  goals. 

Expenses  to  remain 
within  budgeted  amount 

FY2001;  goal  achieved 
FY2002:  goal  achieved 
FY2003:  not  stated  in  Service 
Standard  Report 

|C.  STAFFIWG  PERFORMAiyCE  ■"  * 

' ' 

Ic.  Not  vncmicins  hy 
positini)  (vaconnios 
rniiinininii  oncn 
prnmntinns  and  nnw 
liirns  havn  hmm 
dndiictml  from  rutirnns 
or  rnsiiinatinns)  for 
onch  division. 

I\lnt  vacancios  hy 
position  (vacancios 
roinaininii  onco 
promattons  and  now 
hiros  havn  haon 
dnductml  from  rotiroos 
or  rosiflnations)  for  oach 
division. 

Budootod  positions  (for 
the  yoar)  and  percent  of 
vacancios  (quarterly  hy 
division) 

MTA  Board  adopted  a 
definition  and  method 
of  measurement  and 
specific  goals. 

^ 5%  vacancy  rate 

FY2001:  goal  achieved 
FY2002:  goal  achieved 
FY2003:  goal  achieved 

2o.  Allrilion  ratos  for  now 
ninplnyoos,  hy  division 
ami  lovol. 

Attrition  ratos  for  now 
umployoQs,  hy  division 
and  lovol. 

tf  of  now  hires,  roloases 
and  voluntary  soparations 
divided  by  total  number  of 
now  hiros  in  samo  quarter 
(quarterly  by  division) 

MTA  Board  adopted  a 
definition  and  method 
of  measurement  and 
specific  goals. 

^ 10% 

FY2001:  failed  to  achieve  goal 
FY2002:  failed  to  achieve  goal 
FY2003:  failed  to  achieve  goal 

|D.  CUSTOMER  SERVICE 

Id.  novnlo|ininnl  of 
aniuinl  markotinti  plan 
idniitilyiiid  spodlic 
piopiams  and  prajocts 
that  will  promoto 
incroasod  pntronnpo. 

Hovalopmont  ot 
aniutnl  markotinn  plan 
idnnttfyino  specific 
proprams  and  projocts 
that  will  promoto 
incroasod  patronapo. 

Dovolop  and  publish 
Marketing  Plan  annually. 

MTA  Board  adopted  a 
definition  and  method 
of  measurement  and 
specific  goals. 

Annual  Marketing  Plan 
published 

FY2001:  goal  achieved 
FY2002:  goal  achieved 
FY2003:  not  stated  in  Service 
Standard  Report 
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Service  Standard 

Prop  E Requirements 

Miasurement 

Commants 

Goal  or  Milaitona 

Systemwida  Performance  1 

(Audited  for  FY2001  and  1 

FY2002;  Unaudited  for  FY2003)  | 

2d.  Publication  and 
distribution  to  the 
public  of  schedules 
for  all  trips  taken 
by  all  vehicles 
that  shall  consist 
of  specific  arrival 
times  at  terminals 
and  established 
intermediate  points. 

Publication  and 
distribution  to  the  public 
of  schedules  for  all  trips 
taken  by  all  vehicles  that 
shall  consist  of  specific 
arrival  times  at  terminals 
and  established 
intermediate  points. 

Printing  and  distribution  of 
time  tables  for  all  vehicle 
trips,  consisting  of  specific 
arrival  times  at  terminals 
and  intermediate  time  points 

MTA  Board  ndoptod  a 
definition  and  method 
of  measurement  and 
specific  goals. 

Printing  and  distribution 
of  time  table 

FY2001;  gonlachiovod 
FY2002;  goal  aebiovod 
FY2003:  not  stated  In  Sorvico 
Standard  Report 

3d.  Operator  conduct 
complaints. 

Operator  conduct 
complaints. 

- % of  Passenger  Service 
Reports  resolved  within  30 
days 

- % change  in  H of 
Passenger  Service  Reports 
from  previous  yearlby  type 
of  complaint) 

MTA  Board  adopted  a 
definition  and  method 
of  moasuromont  and 
specific  goals. 

Goal  1:  75%  of  all 
Passenger  Service 
Reports  resolved  in  30 
days 

Goal  2:  10%  reduction 
of  Passenger  Service 
Ruperts  annually  (10% 
reduction  from  previous 
year) 

FY2001:  goal  achieved  lor 
complaint  resolution;  lailod  to 
nchiovo  goal  for  II  of  l*SRs 
FY2002:  failed  to  achiovo  goals 
FY2003:  goal  achieved  lor 
complaint  rosolutiun;  lulled  to 
achieve  goal  for  II  ol  PSRs 

4d.  Annual  passenger 
surveys  and  follow-up 
by  management. 

Annual  passenger 
surveys  and  follow-up  by 
management. 

- Passenger  Survey 
conducted 

- Employee  Survey 
conducted 

MTA  Board  adopted  a 
definition  and  method 
of  measurement  and 
specific  goals. 

Conduct  annual 
passenger  survey  end 
employee  survey 

FY20G1:  goal  aebiovod 
FY2002:  goal  achioved 
FY2003:  not  stated  in  Service 
Standard  Report 

5d.  Improvements  in  public 
information  regarding 
vehicle  delays  during 
operations  as  well 
as  any  general  user 
information  regarding 
system  modifications, 
route  changes  and 
schedules. 

Improvements  in  public 
information  regarding 
vehicle  delays  during 
operations  as  well 
as  any  general  user 
information  regarding 
system  modifications, 
route  changes  and 
schedules. 

Was  not  (fbvetoped  in 
FY2001  or  FY2002  (in 
FY2004,  staled  that 
measurement  would 
'Assess  current  practices, 
develop  and  implement 
improvement  plan.') 

Did  not  develop  specific 
measurement  until 
FY2004. 

Feedback  from  passenger 
surveys 

Rider  surveys  were  conducted, 
but  wore  not  used  to  assess  tho 
performance  of  this  measuro. 
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Service  Standard 

Prop  E Requirements 

Measurement 

Comments 

Goal  or  Milestone 

Systemwide  Performance 
(Audited  for  FY2001  and 
FY2002;  Unaudited  for  FY2003) 

6(1.  Efforls  In  improvo 
[Jrivnr  Irfiintnu, 
tochnicnl  as  wrjil  as 
acci(l(jnt  follow-up. 

Efforls  lo  improve  driver 
traininp,  lechnical  as 
well  as  accident  follow- 
up. 

- //  of  hours  and  type  of 
Driver  Training  (quarlerly 
and  FYTD). 

- //  of  accidents,  number  of 
operator  pay  hours  worked, 
accident  rale  per  100,000 
operator  pay  hours  worked 
(systemwide,  division  and 
year 

- total  number  of  accidents 
also  reported  quarterly). 

MTA  Board  adopted  a 
definition  and  method 
of  measurement  and 
specific  goals. 

FY2001  and  FY2002: 

100.000  operator 
training  hours 
FY2003and  FY2004: 

50.000  operator  training 
hours 

5%  accident  reduction 
over  previous  year 

Goals  for  operator  training  hours 
were  achieved  in  all  three  fiscal 
years. 

Failed  to  achieve  accident 
reduction  goals  for  all  three  years. 

7(1.  Nuinhor  of  criiiio 
inokloiils  on  Miinicipnl 
Itailwny  vohtcins  or 
in  Miinicipnl  Hnilwny 
fncilitins. 

Niimlior  of  crimo 
inctdonts  on  Municipal 
ftnilway  vohiclos  or 
in  Miinicipnl  nailway 
fncilitios. 

//  of  incident  reports  on 
Muni  vehicles  or  property 
(systemwide  by  quarter  and 
anntinlly) 

MTA  Board  adopted  a 
definition  and  method 
of  measurement  and 
specific  goals. 

5%  reduction  from 
previous  year 

Goal  was  achieved  in  all  three 
years. 

E.  EMPLOYEE  SATISFACTION  . . ' ' ' ' • 1 

1n.  Mtimlim  of  nrinvancos. 

Ntinihor  of  oriovnnces. 

If  of  employee  grievance 
reports  for  transit  operators 
and  maintenance  8i  misc. 
employees  (quarterly  and 
annually). 

MTA  Board  adopted  a 
definition  and  method 
of  measurement,  but 
no  specific  goal  was 
adopted. 

Quarterly  grievance 
report 

No  specific  goal  was  adopted. 
Grievance  reports  were  published 
quarterly. 

2o.  Spoml  uf  rnsoliition  of 
liriovniicns. 

Spninl  of  rosoliition  of 
priovanros. 

Percent  of  Operations  and 
Maint.  & Misc,  Employees 
grievances  resolved  within 
30  days. 

1 

MTA  Board  adopted  a 
definition  and  method 
of  measurement  and 
specific  goals. 

Resolve  75%  of  internal 
grievances  within  30 
days 

For  all  quarters  in  which  data 
were  reported,  the  goal  was 
achieved  for  operator  grievances. 
Achievement  was  less  consistent 
for  maint.  and  misc.  employee 
grievances. 
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\ Service  Standard 

Prop  E Requirements 

Measurement 

Comments 

Gnil  nr  MilntttfinA 

Systamwida  Parformanca 
(Audited  for  FYZOOIand 

3e.  Longevity  of 
employment. 

Longevity  of 
employment. 

It  of  employees  that  leave 
through  retirements, 
terminations  and  promotions 
(job  category,  annual) 

MTA  Board  adopted  a 
definition  and  method  of 
measurement,  however 
no  specific  goals  for 
longevity  of  employment 
wore  adopted. 

Annual  report  of  longevity 
of  omploymenl 

Reports  wore  publishod  in  FY2001 
and  FY2002. 

4e.  Employee  recognition. 

Employee  recognition. 

Tracking  of  all  employee 
award  programs. 

MTA  Board  adopted  a 
definition  and  mothod  of 
measurement,  however 
no  specific  goals  for 
longevity  of  employment 
wore  adopted. 

Annual  achiovoment  of 
honoroos  in  the  following 
award  programs  {It  of 
awards); 

(12)  Systemwide 
Operators  of  the  Month 
Award 

(4)  Transit  Su|iorvisors  of 
the  Quarter  Award 
(4)  Finance  & Admin 
Employees  of  the  Quarter 
(12)  Maintenance 
Employee  of  the  Month 
(4)  Maintenance 
Supervisors  of  the 
Quarter 

(4)  Safety  & I raining 
Employee  of  the  Quarter 
(4)  Accessibility  Employee 
of  the  Quarter 

Actual  luimbor  of  awards  mado 
wore  not  indiidod  in  iho  Sorvico 
Standards  Roports. 

5e.  Employee  education  and 
training  opportunities. 

Employee  education  and 
training  opportunities. 

If  of  hours  of  training 
classes  by  type  of  training. 

MTA  Board  adopted  a 
definition  and  method  of 
measurement,  however 
no  specific  goals  for 
longevity  of  employment 
were  adopted. 

50,000  annual  hours 
of  employee  training  or 
approximately  20  liours 
per  FTE 

FY200I:  goal  achieved 
FY2002:  goal  achieved 
FY2003;  failed  to  achieve  goal 

San  Francisco  Municipal  Railway 

An  Independent  Transportation  Quality  Review 

This  biennialTransportation  Quality  Review  mandated  by  Proposition 
E provides  yet  another  tool  that  Muni  can  use  to  continue  to  improve 
its  performance.  Tiiis  review  has  been  conducted  with  the  following 
goals  in  mind: 

• Hcl[j  MiM)i  assess  its  progress  toward  the  goals  and  objec- 
tives of  Proposition  E and  deveiop  action  plans  for  enhancing 
performance  in  areas  wlicre  there  tnay  be  some  deficiency. 

• Evaiiiatc  Muni's  established  goals  and  performance  against  the 
letter  and  intent  of  Proposition  E. 

• Assess  whether  specific  implementation  goals  and  methods 
and  definitions  of  measurement  arc  appropriate  or  could  be 
improved. 

• Provide  inde|)endcnt  verification  to  the  public  that  Muni  is 
on  track  by  auditing  Muni's  data  collection  and  analysis  pro- 
cedures. 

This  Transportation  Quality  Review  consists  of  four  main  elements; 

• Data  review  and  verification  of  performance  - Propo- 
sition E ro(|uircs  a routine  audit  of  Muni’s  quality  assurance 
|)i  ocess  Including  an  audit  of  data  collection  metliods  and 
service  standard  reporting.  This  is  the  first  audit  of  Muni's 
reporting  on  service  standards  and  covers  fiscal  years  2001 
and  2002  (July  I,  2000  - June  30.  2002).  Auditors  reviewed 
Muni's  Service  Standards  Reports  from  those  years  to  verify 
that  data  were  collected  according  to  the  definitions  and 
methods  of  measurement  specified  by  the  MTA  and  that 
reportetl  service  standards  were  computed  correctly.  Sys- 
tematic spot  checks  of  original  source  data  and  of  automated 
tracking  systems  and  procedures  were  used  to  determine  the 
accuracy  of  reported  service  standards.  During  the  months 
of  October  and  November  2003.  auditors  met  with  Muni  staff 


responsible  for  data  collection  and  computation  to  review 
both  data  collection  and  computation  procedures  as  well  as 
the  actual  performance  data.  Appendix  A lists  the  Muni  staff 
who  were  interviewed. 

Trends  analysis  - Auditors  reviewed  trends  in  data  and 
performance  achievement  over  the  two-year  audit  period, 
as  well  as  unaudited  data  and  performance  from  fiscal  years 
2003  and  2004.  Findings  from  this  trends  analysis  were  used 
to  develop  Action  Plans  for  those  areas  in  which  Muni’s  per- 
formance could  be  improved. 

Auditor  recommendations  and  Action  Plans -Auditor 
recommendations  focus  on  ways  to  further  refine  or  improve 
performance  reporting  to  make  it  more  relevant  to  the  MTA. 
Muni  management  and  the  public.  Action  Plans  are  similar  in 
nature  to  auditors'  more  general  recommendations,  however 
they  focus  more  specifically  on  ways  to  improve  performance 
or  make  reporting  more  accurate  and  relevant,  particularly 
for  those  service  standards  in  which  Muni  has  failed  to  meet 
its  performance  goals.  Although  the  recommendations  and 
Action  Plans  focus  on  the  two-year  audit  period,  they  incor- 
porate any  changes  that  have  been  made  since  that  time.  Both 
the  recommendations  and  Action  Plans  were  reviewed  with 
Muni  staff  to  ensure  that  all  recommendations  were  in  line 
with  current  budget  and  resource  constraints. 

Documentation  and  communication  of  results  -This 
report  contains  the  results  of  the  audit,  as  well  as  all  auditor 
findings,  recommendations  and  Action  Plans.  Additionally,  a 
more  reader-friendly  "Report  Card"  will  be  developed  that 
will  summarize  performance  trends  and  recommendations  in 
easy-to-understand,  lay  terms. 
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Key  Findings  and  Recommendations 

Overall.  Muni  has  done  an  excellent  job  of  fulfilling  its  Proposition 
E mandate.  Auditors  found  that  data  were  accurately  recorded  and 
reported,  and  that  reports  were  made  available  to  the  public  Ind 
reviewed  with  the  Citizens’  Advisory  Committee  and  MTA  Board 
quarterly.  Performance  was  generally  good  and  improved  over  time; 
although  there  are  some  areas  that  will  require  specific  attention  in 
order  to  achieve  established  performance  goals. 

The  auditors  also  identified  a number  of  opportunities  to  further 
streamline  and  improve  the  Service  Standards  in  order  to  make  them 
more  relevant  to  the  public,  the  MTA  Board  and  Muni  management. 
Data  do  not  always  yield  the  type  of  information  that  decision  makers 
can  use  to  develop  plans  and  actions  that  will  enable  improvements 
in  performance.  Our  recommendations  and  Action  Plans  focus  on 
ways  that  Muni  can  amend  definitions  or  methods  of  measurement 
and  performance  goals  so  that  the  service  standards  can  provide 
information  that  will  serve  as  a truly  useful  management  tool.  In 
some  cases,  we  recommend  eliminating  service  standards  that  fail  to 
provide  truly  useful  information,  since  there  is  no  point  in  collecting 
data  if  it  cannot  be  transformed  into  useful  information. 

Following  is  a summary  of  the  key  findings  and  recommendations  of 
this  report: 

A.  System  Reliability 

To  improve  on-time  performance  and  meet  the  goals  es- 
tablished under  Service  Standard  la,  it  will  be  necessary  to 
reduce  the  incidence  of  early/sharp  operations.  It  may  also 
be  necessary  to  make  schedule  and  route  revisions  on  those 
lines  that  have  chronically  poor  performance. 

For  Service  Standards  la  and  6a,  it  may  be  advantageous  to 
reallocate  resources  used  to  perform  terminal  checks  so  that 
a greater  number  of  line  checks  can  be  made.  The  goal  of  such 
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a reallocation  would  be  to  increase  the  frequency  of  checks  in 
order  to  provide  additional  seasonal/quarterly  observations 
on  each  mode.  This  does  not  suggest  that  terminal  leaving 
times  are  unimportant  to  reliability,  but  rather  that  counts 
could  be  more  effectively  taken  elsewhere  on  the  line. 

Foi  Set  vice  Standards  la  and  6a,  Muni  should  consider  measur- 
ing on-f/me  performonce  (schedule  adherence)  only  for  those 
services  with  headway  intervals  of  more  than  10  minutes,  and 
measure  heodwoy  od/ierence  (time  between  transit  vehicles) 
on  chose  services  with  headway  intervals  of  10  minutes  or 
Icss. 

In  order  to  achieve  established  goals  for  die  porcoiuagc  of 
scheduled  service  hours  that  are  delivered  in  Service  Standard 
2a,  it  will  bo  necessary  to  address  tlio  loss  of  service  due  to 
operator  unavailability.  It  would  be  prudent  for  Operations 
Manager  s to  draft  a [)lan  to  consiflor  reallocatiot)  of  o[)orator 
resources  between  modes  in  order  to  equalize  any  missed 
service  systemwide.  It  is  probable  that  specific  shortages  in 
operator  availability  In  any  single  mode  will  not  bo  addressed 
in  a systemwide  target  but  should  bo  minimized  on  a mode 
and  division  basis.  NoiwIthstaiuJIng  labor*  agr  ocmonis  and  past 
practices,  Operations  Managers  should  take  all  necessary  steps 
to  maintain  the  higficst  levels  of  on-street  service  pr  ovision 
when  there  arc  spikes  in  operator  availability. 

• Service  Standard  3a  should  be  eliminated  since  it  is  simply  the 
inverse  of  Service  Standard  2a  and  is  completely  duplicative. 

• Service  Standard  -la  should  be  coordinated  with  Service 
Standard  5a  so  that  all  lines  identified  as  exceeding  85%  of 
capacity  riuring  peak  periods  arc  monitored  as  part  of  Service 
Standard  4a.  Additional  data  available  througfi  tfre  Passenger 
Service  Reports  database  should  be  utilized  to  conduct  fol- 
low-up traffic  checks  on  lines  chronically  identified  as  having 
pass-ups  or  peak  period  loads  that  exceed  85%  of  capacity. 
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• For  Service  Standard  6a,  the  method  of  measuring  headway 
adlierence  should  be  modified  so  that  it  is  more  relevant  to 
high  frequency  service  (headway  intervals  of  10  minutes  of 

less). 

B.  System  Performance 

• For  Service  Standard  I b.  realistic  goals  for  ridership  should  be 
established  based  on  economic  conditions  as  well  as  available 

service. 

• Fare  revenue  is  an  important  management  tool  that  can  serve 
as  a surrogate  for  ridership  during  periods  of  fare  stability. 
Coml)inod  with  otlier  indicators,  it  is  possible  to  calculate 
average  fare  per  passenger,  a useful  management  tool  when 
looking  at  alternative  fare  structures.  Wile  this  indicator 
should  be  continued,  the  current  practice  of  changing  the 
l)('iichmaii<  for  fare  revenue  annually  based  on  the  prior  year's 
actual  experience  docs  not  allow  for  reasonable  comparison 
across  time.  Rather  than  continuing  to  alter  the  benchmark, 
it  is  recommended  that  the  milestone  be  rcstated:**Fare  rev- 
enue shoulil  increase  by  1.5%  per  year  compared  with  prior 
year  performance."  Changing  the  milestone  in  this  way  will 
allow  for  comparisons  over  time,  and  will  serve  as  a better 
indicator  of  the  need  to  adjust  fares  and  fare  structure. 

• Miles  and  hours  of  service  are  a direct  reflection  on  the  avail- 
able budget.  It  is  not  reasonable  to  expect  that  in  times  of 
budget  constraints  miles  and  hours  of  service  will  continue  to 
increase,  or  even  remain  constant.  Miles  aitd  hours  of  service 
arc  important  inputs  to  the  NationalTransit  Database  (NTD) 
i*e|^orting,and  will  be  calculated  annually  whether  or  not  they 
continue  to  be  used  as  a performance  measure.  However, 
it  is  recommended  that  this  NOT  be  continued  as  a service 
standard  at  this  time. 


• The  goal  and  information  presented  for  Service  Standard  4b 
has  limited  value  to  the  public  or  to  Muni  itself.  In  fact,  Muni 
has  no  opportunity  to  exceed  its  budget  without  receiving 
a supplemental  appropriation,  as  a City  agency  cannot  oper- 
ate in  deficit.  Rather  than  comparing  expenses  to  budget  by 
mode,  which  is  an  artificial  comparison,  it  is  recommended 
that  Muni  track  fully  allocated  costs  per  hour  of  service  by 
mode.  This  standard  industry  benchmark  would  allow  Muni 
to  compare  its  unit  costs  against  other  similar  properties 
nationwide.  This  information  is  calculated  by  the  Finance 
Department  regularly  for  the  NTD  report,  and  would  not 
require  additional  effort. 

Staffing  Performance 

• For  Service  Standard  I c,  the  great  majority  of  vacancies  can- 
not be  filled  due  to  budgetary  policies.  In  order  to  ensure 
that  service  provision  does  not  suffer,  Muni  has  focused  on 
making  sure  that  maintenance  and  operator  positions  are 
filled.  Consequently,  the  auditor  recommends  that  this  Service 
Standard  be  limited  to  Transit  Operators  and  Maintenance 
personnel,  or  that  the  goals  for  other  divisions  (exclusive  of 
Transit  Operators  and  Maintenance  personnel)  be  revised  so 
that  they  be  reasonably  met  given  budgetary  constraints. 

• In  order  to  track  the  attrition  rate  for  new  hires  in  Service 
Standard  2c,  attrition  rates  should  either  be  reported  at  the 
end  of  new  employees’  probationary  periods  or  within  one 
year  after  hire.  To  calculate  attrition,  only  those  employees 
who  separate  after  six  months  or  one  year  of  service  should 
be  counted.  Furthermore,  there  is  no  information  provided  in 
this  measure  about  why  employees  are  voluntarily  separating 
froni  Muni  service.  If  it  is  possible  to  obtain  this  information 
from  the  Position  Control  System  database,  then  it  should 
also  be  summarized  and  included  in  this  measurement. 
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D.  Customer  Service 

For  Service  StancJard  I d.  marketing  priorities  should  be  made 
within  budget  constraints  and  with  the  goal  of  increasinj; 
patronage:  data  from  ridership  surveys  should  be  used  to 
evaluate  the  success  of  marketing  efforts. 

• For  Service  Standard  2d,  the  current  process  of  schedule 
preparation  is  cumbersome,  time  consuming,  and  relies  on 
the  background  and  knowledge  of  one  key  staff  person.  The 
capabilities  of  the  new  Trapeze  software  should  be  fully  utilized 
in  order  to  simplify  and  streamline  the  process  of  schedule 
preparation. 


For  Service  Standard  3d,  operator  conduct  complaints  need 
to  be  more  clearly  measured  and  reported,  realistic  goals  for 
reducing  the  number  of  operator  conduct  complaints  should 
be  established,  and  the  efficiency  and  accuracy  of  complaint 
infol  niation  uptake  should  be  improved. 

To  be  truly  useful.  Service  Standard  5d  should  be  divided  into 
two  goals  to  distinguish  between  routine  information  provided 
in  real  time  and  longer-term  information  provided  as  part  of 
SCI  vice  or  schedule  change  and/or  construction  projects.  To 
further  differentiate  between  immediate  information  provided 
to  passengers  on  the  system  during  del.ays  or  other  unusual 
events,  and  longer-term  information  provided  to  address  con- 
struction or  other  longer-term  events  the  following  systems 
of  measurement  are  recommended: 


^oal; 

Purpose: 

Definition  of  Measurement: 

Improve  Real-Time  Passenger  Information  regarding  service  delays  ' — i 

^ovide  the  public  with  information  fegarding  service  delays  tlial  will  onnblo  Ilium  lo  make  choices  ahoiil  their  trin 

itZr,'hpTo7  by  communication  of  service  prohleins  and  other  to  each  vehicle,  the  station 

p atforms,  the  Telephone  information  Center,  media,  the  Service  Hotline,  and  Service  Quality  Teams. 

Operators  will  make  passenger  announcements  in  all  cases  where  service  delays  exceed  3 minutes  in  the  suhway  and  1U 
minutes  on  surface  lines.  r lu 

LRv'lin'es'’""'’'  ‘'“'ays  3 tninutos  on  all 

Service  Quality  Teams  will  be  assigned  to  alt  unscheduled  service  disruptions  exceedimi  two  limirs 

ivietnod  of  Measurement: 

Central  Control  logs.  

Passenger  Satisfaction  Surveys. 

fviiiestones: 

95%  of  all  delays  of  3 minutes  or  more  reported  via  station  address  systern.Service  Quality  Teams  assigned  lo  all 
unscheduled  service  disruptions  exceeding  2 hours. 

75%  of  all  passengers  returning  surveys  indicate  that  information  regarding  service  delays  is  adequate  and  useful 
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• In  order  to  meet  the  accident  reduction  goals  established  in 
Service  Standard  6d,  it  will  be  necessary  to  develop  a compre- 
hensive approach  and  coordinated  effort  to  reduce  accidents 
across  divisions  and  departments.  Driver  training  was  found 
to  bo  sufficient.  Therefore,  the  focus  of  accident  reduction 
efforts  should  be  enforcement  of  safety  rules  and  procedures 
and  accident  prevention  through  thorough  review  of  all  ac- 
cidents. 

E.  Employee  Satisfaction 

• In  addition  to  the  Employee  Satisfaction  Service  Standards, 
Muni  is  also  measuring  employee  satisfaction  via  its  annual 
Employee  Survey.  The  results  of  this  survey  should  be  incor- 
porated into  tills  group  of  service  standards  since,  in  many 
respects,  it  provides  a better  measure  of  employee  satisfaction 
than  the  current  group  of  service  standards. 

• For  Service  Standard  I e.  a goal  or  benchmark  for  the  number 
and  type  of  grievances  should  bo  established  so  that  man- 
agement can  more  effectively  monitor  changes  in  grievance 
rates. 

• For  Service  Standard  2c.  the  method  for  reporting  speed  of 
resolution  of  grievances  for  non-opei  ator  grievances  needs  to 
he  closely  examined  and  monitored  to  ensure  that  information 
is  reported  correctly  in  the  future. 

• Service  Standard  3c  provides  almost  no  information  about 
employee  satisfaction.  If  anytliing,  it  is  more  a measure  of 
staffing  performance,  since  long-term  employees  (and  relatively 
minimal  staff  turnover)  reduce  the  need  to  hire  and  train 
now  employees.  The  audito?'  therefore  recommends  that  this 
measure  he  eliminated  from  the  Service  Standards  Report,  or. 
if  retained,  moved  to  the  Staffing  Performance  section. 


For  Service  Standard  4e,the  actual  number  and  type  of  annual 
and  quarterly  awards  made  should  be  included  in  the  Service 

Standards  Report. 

For  Service  Standard  5e,  all  employee  training  hours  should 
be  tracked  in  some  type  of  centralized  system  in  order  to 
facilitate  future  reports  and  audits. 
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A.  System  Reliability 

This  group  of  service  standards  measure  system  reliability  — the 
extent  to  which  transit  vehicles  arrive,  depart  and  are  able  to 
accommodate  all  passengers  at  expected  times  and  frecjuencies. 
System  reliability  is  the  most  significant  factor  in  attracting  choice 
riders,  as  riders  are  unlikely  to  depend  on  a system  that  is  unreliable. 
Poor  system  reliability  can  significantly  lengthen  the  amount  of  time 
passengers  spend  both  waiting  for  a transit  vehicle  and  traveling  to 
their  final  destination.  It  can  also  result  in  crush  loads  and  pass  ups, 
where  transit  vehicles  are  at  capacity  and  unable  to  accommodate 
waiting  passengers.  This  situation  often  leads  to  further  delays  and 
breakdowns  in  service  reliability.' 

Proposition  E specified  the  following  service  standards  in  order 
to  improve  and  maintain  the  reliability  of  Muni  transit  service. 
Collectively,  these  performance  measures  give  on-going  snapshots 
and  the  ability  to  diagnose  service  delivery  problems. 

la.  % of  vehicles  that  run  on  time  according  to  published 
schedules  (no  more  than  4 minutes  late  or  I minute  early) 
measured  at  terminals  and  established  intermediate  points. 

• 2a.  % of  scheduled  service  hours  that  are  delivered  and 
percentage  of  scheduled  vehicles  that  begin  service  at  the 
scheduled  time. 

3a.  Missed  service  due  to  either  insufficient  vehicles  or  driver 
unavailability  as  a percentage  of  scheduled  service  hours.  (This 
is  simply  an  inverse  of  the  preceding  service  standard.) 

4a.  % of  vehicles  that  pass  published  time  points  during 
measurement  periods  unable  to  pick  up  passengers  due  to 
crowding  without  being  followed  within  3 minutes  or  less  by 
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another  vehicle  on  the  same  route  with  space  for  all  waiting 
passengers. 

5a.  Peak  period  passenger  load  factors. 

6a.  Actual  headways  against  scheduled  headways  on  all  radial, 
express,  cross-town,  secondary,  and  feeder  lines  for  peak,  base, 
evening,  and  late  night  services. 

7a. % vehicle  availability  and  reliability  (moan  distance  between 
failure)  by  mode. 

8a.  Unscheduled  absences  by  operator,  mechanical  and  admin- 
istrative personnel. 

9a.  Milos  between  roadcalls  by  mode. 
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Key  Recommendations 

The  following  arc  key  recommendations  based  on  the  findings  of  our 
audit  of  FY200I  and  FY2002  Service  Standards  data  and  on  trends 
observed  in  unaudited  FY2003  and  FY2004  Service  Standards  data: 

• To  improve  on-time  performance  and  meet  the  goals  es- 
tablished under  Service  Standard  la,  it  will  be  necessary  to 
reduce  the  incifloncc  of  carly/sharp  operations.  It  may  also 
be  necessary  to  tnakc  schedule  and  route  revisions  on  those 
linos  that  have  chronically  poor  performance. 

• For  Service  Standards  la  and  6a.  it  may  be  advantageous  to 
reallocate  resources  used  to  perform  terminal  checks  so  that 
a greater  number  of  line  diecks  can  be  made.  The  goal  of  such 
a reallocation  would  be  to  iticreasc  the  frequency  of  checks 
to  [)rovide  additiotial  scasonal/quarterly  observations  on  each 
mode.  This  does  not  suggest  that  terminal  departure  times 
are  unim[)ortant  to  relial)ility,  l)ut  rather  that  counts  could  be 
more  effectively  taken  elsewhere  on  the  line. 

• For  Service  Standards  I a and  6a.  Muni  should  consider  measur- 
ing on-tlmc  performance  (schedule  adherence)  only  for  those 
services  with  headway  intervals  of  more  than  1 0 minutes,  and 
measure  headway  adherence  (time  between  transit  vehicles) 
on  those  services  with  headway  intervals  of  10  minutes  or 
loss. 

• In  order  to  achieve  established  goals  for  the  percentage  of 
scheduled  service  hours  that  are  delivered  in  Service  Standard 
2a.  it  will  he  necessary  to  address  the  loss  of  service  due  to 
operator  unavailability.  It  would  be  prudent  for  Operations 
Managers  to  draft  a plan  to  consider  reallocation  of  operator 
resources  between  modes  in  order  to  equalize  any  missed 
service  systemwide.  It  is  probable  that  specific  shortages  in 
operator  availability  in  any  single  mode  will  not  bo  addressed 
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in  a systemwide  target  but  should  be  minimized  on  a mode 
and  division  basis.  Notwithstanding  labor  agreements  and  past 
practices,  Operations  Managers  should  take  all  necessary  steps 
to  maintain  the  highest  levels  of  on-street  service  provision 
when  there  are  spikes  in  operator  availability. 

Service  Standard  3a  should  be  eliminated  since  it  is  simply  the 
inverse  of  Service  Standard  2a  and  is  completely  duplicative. 

Service  Standard  4a  should  be  coordinated  with  Service 
Standard  5a  so  that  all  lines  identified  as  exceeding  85%  of 
capacity  during  peak  periods  are  monitored  as  part  of  Service 
Standard  4a.  Additional  data  available  through  the  Passenger 
Service  Reports  database  should  be  utilized  to  conduct  fol- 
low-up traffic  checks  on  lines  chronically  identified  as  having 
pass-ups  or  peak  period  loads  that  exceed  85%  of  capacity. 

For  Service  Standard  6a,  the  method  of  measuring  headway 
adherence  should  be  modified  so  that  it  is  more  relevant  to 
high  frequency  service  (headway  intervals  of  10  minutes  or 
less). 
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la.  /o  of  vehicles  that  run  on  time  according  to  published  schedules  (no 

more  than  4 minutes  late  or  I minute  early)  measured  at  terminals  and 
established  intermediate  points. 


Goal: 

No  less  than  85%  on  July  1,  2004 

Purpose: 

To  measure  schedule  adherence 

Definition  of 
Measurement: 

FY2001:  Monthly  measurement  of  the  5 most  patronized  routes  for  each  division.  Such  chocks 
shall  be  conducted  no  less  often  than  10  weekdays  and  weekends  per  month  on  5 difforont  roiitos 
per  month  in  each  division,  beginning  with  the  most  patronized  route,  progressing  in  tho  order  of 
declining  route  patronage,  and  ending  with  the  least  patronized  route.  The  cycle  shall  thou  he 
repeated. 

FY2002:  Each  line  will  be  checked  twice  a year.  Such  checks  shall  bo  conducted  no  loss  ollon 
than  10  weekdays  and  weekends  per  check.  An  annual  chockiiin  schedule  shall  ho  osinhiishod  lor 
the  routes.  The  order  in  which  the  routes  are  checked  will  bo  detorminod  monthly  lhrou(|li  a random 
selection  process.  To  the  extent  automated  systems  can  bo  substiliitod  at  loss  cost  for  such 
checks,  or  the  measurement  of  any  performance  standard,  such  systems  must  bo  used. 

Method  of 
Measurement: 

Check  the  designated  lines  using  criteria  of  -1  minute  to  ^4  minutes.  "Periods  of  lime  includes 
morning  rush  (6  a.m.  to  9 a.m.)  midday  (9  a.m.  ■ 4 p.m.l  evening  rush  (4  p.m.  to  7 p.m.)  and  night 
(7p.m.  to  1 a.m.l.  Supervisory  personnel  shall  conduct  a one  hour,  on-time,  and  load  standard  chock 
at  a maximum  load  point  at  mid-route  during  all  four  time  periods  stated  above." 

Milestones: 

FY2001  FY2002 

65%  70% 

From  fAuru  Standards  Rtport,  FY200I  and  fY2002 
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Reported  Performance 

Table  A- 1 summarizes  the  quarterly  results  by  mode  for  fiscal  years 
200 1 2002, 2003  and  the  first  quarter  of  fiscal  year  2004.  The  jump  in 
performance  for  the  light  rail  and  cable  car  modes  between  the  third 
and  fourth  quarters  of  FY200I  was  due  to  a change  in  measurement 
procedure.  Beginning  in  the  fourth  quarter  of  FY200I,  scheduled 
arrival  times  were  no  longer  linked  to  scheduled  run  nunnbers 
for  cable  car  and  light  rail  modes.  Instead,  checks  were  made  to 
determine  whether  any  vehicle,  rather  than  a specific  run  number, 
arrived  at  scheduled  times.  (This  change  was  approved  by  the  MTA 
Board.)  Tlie  original  FY2002  goal  was  lowered  from  75%  to  70%. 
Consequently,  the  established  goal  was  met  for  both  FY200I  and 
FY2002  for  systemwide,  fourtli  quarter  performance. 

Subsc(|ucnt  to  tlic  audit  period,  on-time  performance  held  steady 
in  FY2003  (unaudited  figures)  but  failed  to  meet  the  75%  on-time 
standard  and  appears  to  have  declined  in  the  first  quarter  of  FY 
2004, 


Table  A- 1 On-Time  Performance,  FY200I  - FY2004 


able  A- 1 un- 1 in 

le  rc» 

1st 

Quarter 

2nd 

Quarter 

3rd 

Quarter 

4th 

Quarter 

FY 

Goal 

rv9nni  On-Time  Performance  ■ 

Motor  Coach 

58.6% 

60.3% 

68.3% 

67.8% 

Trolley 

61.8% 

65.1% 

61.9% 

66.1% 

Lipht  Rail 

42.9% 

45.5% 

42.5% 

63.0% 

65% 

Cable  Car 

18.0% 

18.3% 

17.3% 

67.2% 

Systemwide 

50.2% 

53.4% 

50.6% 

65.5% 

1 cv9nn9  On-Timfi  Performance  1 

Motor  Coach 

65.1% 

69.4% 

68.6% 

69.9% 

Trolley 

71.6% 

76.1% 

71.3% 

76.1% 

Light  Rail 

59.8% 

65.1% 

67.7% 

67.8% 

70% 

Cable  Car 

68.7% 

67.0% 

• • 

70.9% 

Systemwide 

66.9% 

70.9% 

69.5% 

71.9% 

1^^ 

1 FY9nn3  On-Time  Performance  I 

Motor  Coach 

70.4% 

68.8% 

67.2% 

71.3% 

Trolley 

73.1% 

74.6% 

74.6% 

73.5% 

Light  Rail 

64.3% 

66.6% 

66.9% 

69.1% 

75% 

Cable  Car 

69.9% 

72.6% 

68.2% 

67.0% 

Systemwide 

70.1% 

70.7% 

70.0% 

70.9% 

1 FY2004  On-Time  Performanc 

B 

Motor  Coach 

69.4% 

Trolley 

68.8% 

Light  Rail 

62.5% 

85% 

Cable  Car 

66.0% 

Systemwide 

67.3% 

From  Mun.  Seroro  StondorH  Ktpen  4th  Quarter  FY200I.  FY2002.  FY2003  and  la  Quortw  FY2004 
Oau  in  lultci  hive  not  bw»  audited 
**  Was  not  measured  this  quaaer. 
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Verification  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

FY200I 

The  following  changes  in  sampling  methodology  were  approved  by 
the  MTA  Board  and  subsequently  implemented  in  FY2001: 

• Light  rail  (LRV)  and  cable  car  divisions  were  combined  into 
one  rail  sampling  rotation  with  results  reported  separately; 
eight  checks  per  month  were  reduced  to  five  or  four  checks 
per  month. 

• Three  motorcoach  divisions  were  combined  into  one  motor- 
coach  sampling  rotation;  1 5 checks  per  month  were  reduced 
to  five  checks  per  month. 

• Two  trolleycoach  divisions  were  combined  into  one  trolley- 
coach  sampling  rotation;  ten  checks  per  month  were  reduced 
to  five  checks  per  month. 

Lines  were  sampled  within  each  mode  according  to  ranked 
patronage  as  described  in  the  definition  of  measurement.  At  least 
ten  weekday  checks  were  performed  during  each  month  for  motor 
coach  and  trolley  coach  modes.  Light  rail  and  cable  car  modes  were 
in  compliance  with  the  definition  of  measurement  for  the  months 
they  were  sampled.  All  time  periods  were  sampled  in  accordance 
with  the  definition  and  method  of  measurement. 

Fr2002 

The  selection  of  routes  was  coordinated  with  the  Data  Collection 
Plan  (DCP),  adopted  by  Muni  to  satisfy  the  annual  NTD  (and 
previously  UMTA/FTA  Section  1 5)  ridership  reporting.  The 
F^200l  methodology  for  sampling  15  lines/checks  per  month  was 
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modified  for  FY2002  to  include  nine  motorcoach  lines,  three  to  four 
trolleycoach  lines,  and  two  to  three  rail  lines.  A random  selection 
process  replaced  the  ranked  ridership  rotation  process.  Specific 
locations  and  time  periods  conformed  to  the  DCP  and  included,  as 
a minimum,  the  Prop  E defined  time  periods. 

Verification  of  Reported  Performance  During  the 
Audit  Period 

Based  on  an  audit  of  the  fourth  quarter  FY200I  and  FY2002 
submittals  by  the  Muni  Analysts  Coordinator,  performance  appears 
to  have  been  reported  correctly  during  the  audit  period. 
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Problems  or  Issues  Identified  During  the  Audit  Perio  „..uod  op  definition  of  measurement,  a more  detailed 

While  there  were  no  specific  problems  or  issues  with  3^alysis  of  the  schedule  adherence  reporting  over  FY200I 

analysis  of  performance  for  specific  lines  identified  ^,<^2idu7rrurs  lines  with  the  poorest  results.  The  following  table  highlights  cumulative 
and  FY2002  shows  some  continuing  trends  in  those  individual  routes/lines  witn  p 

findings  for  tlie  poorest  performing  lines  by  mode: 


Table  A-2  P»n.e.t  Performing  Lines  for  On-Time  Pcrformanc^TlOO^^ 


Motor  Coaoh 
Llnea 

2001 

<60%  on- 
timo 

2001 

<65%  ontima 
>10%  unriy 

2002 

<50%  on-time 

2002 

<70%on-time 

>10%early 

DBX 

X 

15 

X 

17 

X 

X 

X 

19 

X 

X 

2G 

X 

X 

29 

X 

27 

X 

36 

X 

X 

3BAX 

X 

5^ 

X 

X 

X 

X 

76 

X 

X 

BOX 

X 

BIX 

X 

X 

X 

02X 

X 

89 

X 

X 

X 

Trolley  Coach 


<50%  on- 
time 


<65%  on-time 
>10%  early 


2002 

<50%  on-time 


<70%on-time 

>10%early 


14 

X 

- ' 

21 

X 

Light  Rail 
linfls 

2001 

<50%  on- 
time 

2001 

<65%  on-time 
>10%  early 

2002 

<50%  on-time 

2002 

<70%on-time 

>10%early 

49 

X 

J 

X 

X 

X 

K 

X 

X 

L 

X 

X 

M 

X 

X 1 

X 

N 

X 

X 

Cable  Car 
Linas 

2001 

<50%  on- 
time 

2001 

< 65%  on-time 
>10%  early 

2002 

<50%  on-time 

2002 

<70%on-time 

>10%early 

59 

X 

X 

X 

60 

X 

X 

X 

61 

X 

X 

. - 

From  the  FY200I  and  FY2002  Findings  Report  submitted  by  the  Analysis  Coordinator 
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The  systemwide  schedule  adherence  standard  cannot  reasonably  be 
attained  without  specific  corrective  action  to  the  poorest-performing 
lines.  Given  current  trends,  it  is  improbable  that  the  attainment  of 
85%  schedule  adherence  will  be  reached  in  FY2004.  When  examined 
further,  the  auditor  found  that  failure  to  meet  on-time  performance 
was  due  in  large  part  to  early  arrivals.^  This  is  encouraging,  since 
many  delays  in  service  result  from  street  conditions  that  are  beyond 
Muni’s  control.  Early  departures  result  from  many  causes,  and  may 
be  cultural,  behavioral,  or  intentional  - but  most  of  the  causes  are 
entirely  within  Muni’s  control.  If  early/sharp  operation  is  dramatically 
curtailed,  then  achievement  of  85%  on-time  performance  could  be 
possible  for  all  modes. 

In  FY200I,  for  the  81  lines  for  which  on-time  performance  was 
reported,  all  had  a minimum  of  3%  early/sharp  observations,  with  an 
average  of  17.1%  early/sharp  for  motor  coach,  14.9%  early/sharp  for 
trolley  coach,  36.4%  early/sharp  for  light  rail,  and  32.2%  early/sharp 
for  cable  car.  All  of  the  early  arrivals  cited  above  were  at  least  2 
minutes  or  more  ahead  of  schedule. 

In  FY2002,  for  the  80  lines  for  which  on-time  performance  was 
reported,  all  but  one  had  a minimum  of  3%  early/sharp  observations, 
with  an  average  of  18.9%  early/sharp  for  motor  coach,  14.4%  early/ 
sharp  for  trolley  coach,  23.0%  early/sharp  for  light  rail,  and  17.2% 
early/sharp  for  cable  car.  While  systemwide  on-time  performance 
improved  by  14.5%,  the  improvement  in  early/sharp  arrivals  was  less 
dramatic  - only  6.6%.  Additionally,  it  appears  that  for  the  rubber 
tire  combined  motor  coach  and  trolley  coach  modes,  incidence  of 
early/sharp  operation  actually  increased  from  15.6%  to  16.7%.  It 
should  be  noted  that  improvement  in  the  rail  modes  in  both  on- 
time  achievement  and  reduction  of  early/sharp  observations  was 
due  primarily  to  the  change  in  measurement  method  cited  earlier. 


’ Eirfy  is  a sifnAcanc  concnbutor  to  to  mmi  orvun*  p^r^omunc*  kx  othar 

trmsft  i|enoe  u lbid..p.3’46 
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Muni  staff  have  recognized  that  early/sharp  arrivals  are  a significant 
impediment  to  achieving  on-time  performance  goals  and  have  taken 
the  following  steps  in  order  to  address  this  and  otlier  on-time 
performance  problems: 

• Street  Inspectors  will  be  deployed  at  terminals  (to  the  extent 
possible)  to  insist  on  schedule  adherence  and  prevent  early 
departures.  In  order  to  make  the  best  use  of  tl\e  limited  num- 
ber of  Street  Inspectors.  Inspectors  will  now  be  responsible 
for  checking  multiple  modes  at  key  transfer  points. 

• Responsibility  for  on-time  performance  will  be  more  heavily 
focused  at  the  division  level.  Division  Managers  will  have  fewer 
administrative  duties  and  will  bo  able  to  concentrate  more 
on  directly  overseeing  Street  Supervisors. 

• Schedules  will  be  reviewed  to  make  sure  they  properly  coin- 
cide with  running  times. 

• Next  Bus  technology  will  be  used  to  monitor  the  on-time 
performance  of  certain  lines  In  real  time  so  that  Street  Inspec- 
tors can  make  necessary  adjustments  before  more  significant 
on-time  problems  arise. 

• Muni  will  continue  to  work  closely  with  the  Department  of 
Parking  and  Transportation  to  clear  busways  more  f|uickly, 

• Daily  discussions  and  reviews  of  safety  and  on-titne  perfor- 
mance with  operators,  Central  Control,  and  Street  Inspectors 
will  continue.  Muni  staff  will  also  continue  to  meet  with  union 
representatives  in  order  to  obtain  their  support  for  meeting 
on-time  performance  goals. 

• An  education  and  incentive  campaign  is  planned  that  will  in- 
form transit  operators  about  on-time  performance  goals  and 
actual  achievement,  and  will  reward  those  operators  who  have 
best  on-time  performance. 
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Changes  in  Measurement  Since  the  Audit  Period 

Subsequent  to  the  audit  period,  the  MTA  adjusted  the  Definition 
of  Measurement  for  FY2004  by  changing  the  first  sentence  to  read, 
“Each  line  will  be  chocked  at  least  once  in  each  six-month  period.” 

Action  Plan 

Given  the  continued  need  for  on-time  performance  improvement, 
the  recotnmendations  in  this  Action  Plan  focus  on  early  arrivals  and 
on  corrective  actions  for  the  poorest  performing  routes.  The  trends 
in  data  ol)scrved  for  Service  Standards  la  and  6a  (actual  headways 
against  scheduled  headways)  concurrently  require  both  on-street 
attention  to  supervision  and  preparation  of  now  schedules.  The 
modal  matrix  under  “Problems  or  Issues  Identified”  identifies  lines 
with  the  highest  [)riority  for  l)oth  schedule  evaluation/  remediation 
and  supervision. 

liicidcMicc  o(  Fndy/Shni()  Obscivntions 

Attention  should  l)o  directed  to  reducing  the  incidence  of  early/ 
sharp  o[)eration  and  tlicrcby  increasing  dramatically  the  on-time 
performance.  Findings  for  both  audit  years  arc  significant: 


Mnili! 

FY2001 
1 OnTinio 

-■I 

i FY2002 
On  Tiino 

FY2001 

Enrly/Shnrp 

FY2002 

EnrlyfShnrp 

Mnloi  Conch 

G2.R% 

08.4% 

17.1% 

18.9% 

Tiolloy  Conch 

G3.tl% 

73.9% 

1‘1.9% 

14.4% 

MCtlC 

G3.4% 

71.0% 

15.8% 

10.7% 

Ltflhl  Rail 

49.0% 

05.6% 

30.4% 

23.0% 

Cnhio  Cm 

3G.H% 

09.1% 

32.2% 

17.2% 

Systomwido 

9fi.4% 

89.9% 

24.5%  17.9% 
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Given  the  Proposition  E measurement  definition  for  schedule 
adherence  that  allows  a window  of  +I/-4  for  attaining  “on-time" 
status,  those  percentages  of  early/sharp  cited  above  exceeded  the 
+ 1 allowance  by  a significant  amount.  All  such  observations  were 
at  least  2 minutes  or  more  ahead  of  schedule.  To  evaluate  schedule 
adequacy  (i.e.,  to  measure  the  degree  of  appropriate  running  time), 
recovery  time,  and  headway  (capacity),  ahead  of  schedule  (early/ 
sharp)  operation  must  be  curtailed.  Significant  on-street  supervisory 
resources  must  be  deployed  to  monitor  and  manage  both  end 
terminal  departures  and  mid-point  timepoint  schedule  adherence.  It 
is  apparent  that  if  early/sharp  operation  is  curtailed,  then  achievement 
of  85%  on-time  performance  is  within  target  in  all  modes. 
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Schedule  Evaluation/Remediation 

Muni  is  fortunate  to  have  an  in-house  Data  Collection  Plan  that  incorporates  NTD  Passenger  Boarding/Ridership 
sampling  with  Proposition  E system  reliability  performance  sampling.  Muni's  commitment  to  adhere  to  the  Data 
Collection  Plan  enables  periodic  updating  of  boarding  data  for  each  route/line  as  well  as  timepoint-lcvol  running 
time  data.  To  that  end,  new  baseline  data  are  collected  each  quarter  for  several  lines  across  various  modes.  The 
auditor  identified  26  lines  that  incurred  less  than  65%  on-time  adherence  and  greater  than  10%  early/sharp  arrivals. 
The  extent  to  which  those  lines  have  received  baseline  updates  and  analysis  is  as  follows: 


Line 

. FY2002 
, SchAdh% 

Baseline 

PrcFYOl 

i Baseline 
FY01&02 

Baseline 

FY03&04 

Analysis 

Coinpleto 

1 RunninuTimu 
Pfoblum  1 

9BXMC 

70.2 

4/98 

3/04 

4/04 

15 

66.1 

1/96 

2/04 

3/04 

17 

51.3 

8/02 

9/02 

Yes 

19 

69.0 

9/01 

10/01 

No 

26 

57.4 

12/01 

5/02 

Yes 

27 

68.5 

8/02 

10/02 

Yes 

29 

58.6 

1/02 

2/02 

Yes 

36 

62.1 

1/99 

3/00 

No 

38AX 

56.4 

8/02 

9/02 

No 

54 

45.5 

3/93 

4/04 

6/04 

76 

61.7 

BOX 

56.9 

9/03 

2/04 

81X 

39.8 

9/03 

2/04 

82X 

57.0 

9/03 

1/04 

89 

59.0 

2/03 

2/04 

14  TC 

71.5 

12/02 

1/03 

Yes 

21 

73.7 

12/00 

3/01 

Yes 

49 

70.6 

1/03 

1/03 

Yds 

J LRV 

10/00 

3/01 

Yoi 

K 

6/04 

11/04 

L 

3/05 

10/05 

M 

12/04 

7/05 

N 

0/04 

7/04 
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Analysis  of  baseline  data  by  the  Analysis  Coordinator  within  the 
Schedules  Department  has  been  an  on-going  function.  Those  lines 
identified  with  deficient  running  time/recovery  time  have  been 
quantified.  The  required  vehicle  hours  and  peak-to-base  vehicles  to 
implement  a new  schedule  and  remedy  deficiencies  are  as  follows: 


Linn 

Servinn 

Vnhicic 

lIrs/Dny 

' Peak 
Vehicles 

! Base 
Vehicles 

50ay 

Runs 

Pay  Mrs/ 
Day 

2fi  MC 

Wl< 

4 
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No 

1 

29  MC 

Wk 

12 
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Yes 

2 

MC 

Wl< 

1G 

2 

1 

3 

19 

TC 

Wk 

G 

Yos 

Yes 

1 

21  TC 

Wk 

Ifi 

Yes (2) 

Yos 

2 

TC 

Wl< 

24 

3 

2 

3 

28 

Daiiy  Total 

40 

5 

3 

6 

47 

Anmtnl  Total " 

10, 120 

11,891 

From  Ondli^v.t  tubmluod  I7  thoAiudytli  Coordinatoi 
■*'Hi»»pd  on  2S3  wrnkdayi  durlMK  thn  yn.ir 


Annualized  platform  (operator)  costs  for  those  target  lines  listed 
al)ovc  would  he  approximately  $378,000.  Variable  costs  for 
maintenance  and  possibly  capital  costs  for  added  peak  vehicles  have 
not  liecn  estimatetl. 

Houle  [valuation 

Route  length  (one-way  mileage)  and  enrouto  turns/congestion  points 
are  a source  of  on-going  schedule  conflict.  Evaluation  of  each  low- 
[lerforming  route  for  schedule  dcriciencies  should  likewise  include 
route  segment  analysis  to  determine  liow  significant  congestion 
points  contrilntte  negatively  to  schedule  adherence.  If  any  congestion 
point  contributes  more  than  I0'?o  one-way  running  time  variability, 
then  it  is  assumed  that  recovery  time  will  be  insufficient  to  cover 


this  condition  and  enroute  schedule  adherence  will  exceed  the  on- 
time  window.  Several  options  are  available  for  remediation: 

• Add  a "control  point"  mid-route  where  significant  conges- 
tion has  occurred  prior.  The  control  point  should  preferably 
be  at  a major  transfer/generator  location. 

• Consider  segmenting  the  line  where  prevailing  demands  on 
both  ends  are  not  matched  and  the  costs  of  adding  recovery 
time  for  two  new  termini  is  offset  by  the  headway  changes/ 
cycle  time  reduction  on  one  segment. 

• Consider  route  deviations  when  congestion  occurs  at  spe- 
cific locations  during  specific  time  periods. 

• Consider  preferential  right-of-way/transit  lane  designation 
where  specific  congestion  occurs  during  specific  time  periods. 
(All  transit  preferential  signal  treatments,  including  signal  pre- 
emption, should  be  identified). 

Targeted  Improvements 

Implementing  a comprehensive  and  on-going  schedule  revision 
program  is  the  mission  of  all  schedule  departments;  however, 
operational  needs  and  resource  constraints  often  preclude  this 
regularized  approach  to  maintaining  effective  operating  schedules. 
Although  the  specific  lines  identified  previously  as  under-performing 
may  be  prioritized  for  both  supervision  and  new  schedules,  they 
individually  may  not  add  on-time  improvement  sufficient  to  raise  both 
the  modal  and  systemwide  percentage  significantly.  For  example,  the 
motorcoach  89  line  attaining  90%  on-time  status  versus  the  current 
(FY2002)  80?o  would  not  contribute  as  much  as  the  motorcoach  47 
attainment  of  a similar  10%  improvement  (FY2002  on-time  at  55%). 
However,  collectively  the  1 5 under-performing  motorcoach  lines,  if 
improved  up  to  85%,  would  contribute  to  a 6%  improvement  in  the 
mode  from  68.4%  to  74.5%. 
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Prioritizing  light  rail  (LRV)  for  reduction  of  early/sharps  by  1 5%  (from 
23  % to  8%)  would  move  1 .6 1 2 observed  trips  to  on-time  attainment 
and  significantly  raise  the  LRV  mode  to  80.6%.  Similarly,  prioritizing 
cable  car  to  meet  scheduled  headways  (observed  trips  and  not  runs 
currently  are  reconciled  to  scheduled  times)  throughout  the  day 
would  significantly  raise  on-time  attainment  from  69.1%  to  at  least 
85%.  Note  that  the  on-time  window  of  + IM  applies  to  the  day  base 
scheduled  headway  of  8 minutes. 

Sampling  Resources  - 

Reallocation  of  Terminal  Checking  to  Increased  NTD  Checking 

Once  early  departures  from  terminals  is  no  longer  a significant 
problem,  it  may  be  appropriate  to  eliminate  terminal  checks  so  that 
a greater  number  of  line  checks  can  be  performed.  Although  the 
Proposition  E measurement  requirement  included  checks  at  terminals 
and  established  timepoints,  the  checking  resources  expended  to 
collect  from  terminals  is  about  twice  that  required  for  established/ 
intermediate  timepoints.  The  Proposition  E mid-point  sampling 
plans  are  coordinated  with  Munis  NTD  sampling  plans  to  minimize 
the  checking  resources  expended.  The  Data  Collection  Plan  for  FY 
2002/2003  (August  I,  2002)  indicates  that  9,720  hours  per  year  of 
checking  is  required  for  the  Proposition  E program  at  4.5  locations 
per  line.  Additionally,  3,432  hours  per  year  of  checking  is  required 
for  the  NTD  program  at  two  locations  per  line.  Clearly,  a significant 
74%  of  the  combined  programs  is  allocated  to  Proposition  E. 

The  resulting  difference  in  schedule  adherence  (between  terminals 
and  intermediate  timepoints)  is  only  about  I %,  however.  Thus,  while 
checking  terminals  provides  a validation  of  the  checks  at  intermediate 
timepoints  and  identifies  trips  that  leave  the  terminal  off-schedule, 
actual  results  indicate  that  such  a validation  may  not  be  necessary. 
It  would  appear  prudent  to  increase  the  sampling  of  intermediate 
timepoint  checks  in  lieu  of  the  terminal  checks  with  the  goal  of 
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sampling  each  line  each  quarter  rather  than  the  current  twice  yearly 
target.  ' ^ 

The  auditor  recommends  that  the  portion  of  Proposition  E checking 
hours  allocated  to  terminal  checks  (roughly  4,320  hours  per  year) 
be  reallocated  to  mid-point  locations.  The  goal  of  such  a reallocation 
would  be  to  increase  the  frequency  of  checks  to  provide  additional 
seasonal/quarterly  observations  on  each  mode.  Poorly  performing 
routes  should  be  targeted  for  additional  observations  as  well.  If  the 
approximately  4,320  hours  currently  allocated  to  terminal  checks 
were  reallocated  to  the  current  NTD  program  of  3,432  hours,  it 
would  more  than  double  the  frequency  of  checks  made  under  the 
NTD  program. 

Streamlining  the  Service  Standards 

On-time  performance  Is  typically  used  to  nioasure  the  reliability 
of  transit  services  operating  at  headways  that  are  greater  thati  10 
minutes.  For  services  operating  at  headways  of  10  minutes  or  loss, 
headway  adherence  Is  typically  used  to  measure  service  reliability.’ 
From  a passenger’s  perspective,  schedule  adherence  nialtcrs  when 
headways  are  longer  than  10  minutes,  since  passengers  rely  on 
scheduled  arrivals  and  departures  when  planning  their  trips.  For 
headw.iys  that  are  10  minutes  or  loss,  passengers  no  longer  rely  on 
schedules,  since  they  know  a vehicle  will  arrive  shortly  after  they 
arrive  at  their  stop.  Service  reliability  might  be  more  accurately 
measured  by  not  measuring  on-time  performance  for  high  frequency 
(headway  intervals  of  10  minutes  or  less)  transit  services,  but  by 
measuring  headway  adherence  instead  (sec  the  "Action  Plan"  section 
under  Service  Sundard  6a). 


'»xd.p  3-4/ 
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Coordination  with  Other  Goals  and  Objectives 


Coordination  wiin  uuie.  uua,. 

goals  within  this  grot'p  o closely  coordinated  with 

i„pro,e  on.tim=  perlor, nance  particularly 

actions  taken  to  achieve  ot  rer  p delivery  of  scheduled 

those  relaterl  to  headway  adherence  (6a),  delivery 

service  (2a)  and  safety  (6d). 


Use  o(  Cumulative,  Annual  Pcriormance  vs.  Quarterly  Performance 

Pather  than  tisinfi  fourti,  c.narter  results 
, ,|,c  auditor  recommends  that  cumulative  u 

======= 

„ o san,|,led  oath  year  during  the  lourth  qu.artor.  ^ 

to  date  or  cumulative  annual  perlonnance  may  provide  better 
m me  o°  systen.wi.lc  an.l  nio.l.il  pe,  lormance  over  the  entire  year 
'gum  teriy  performance  coi.ld  still  he  used  to  .lete.anine  variations 
!n  progress  tow.ud  meeting  perlonnance  goals  throtighout  • 

Following  are  hoth  the  lourth  .|U.,rter 

systemwide  and  for  individual  modes  in  FY200I  and  FY2002. 
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2a.  % of  scheduled  service  hours  that  are  delivered  and  percentage  of  scheduled 
vehicles  that  begin  service  at  the  scheduled  time. 


Goal: 

No  less  than  98.5%  on  July  1,  2004 

Purpose: 

To  measure  service  hours  through  available  operators  and  available  equipment,  along  with 
the  percentage  of  equipment  available  for  service. 

Definition  of 
Measurement: 

Monthly  measurement  of  the  percent  of  total  available  hours  for  service  measuring 
operators  and  equipment  and  percentage  of  equipment  available  daily. 

Method  of 
Measurement: 

Both  operators  and  equipment  are  measured  as  to  the  total  number  of  hours  in  service 
as  a percentage  of  the  total  scheduled  hours.  Data  come  from  the  online  Dispatching 
System.  Measurement  of  the  vehicles  that  begin  service  at  the  scheduled  time  will  be 
provided  from  the  8 a.m.  and  6 p.m.  'Not-Out  Report*  generated  by  Central  Control  and 
will  show  the  percent  of  vehicles  that  went  out  at  the  scheduled  time  for  both  the  n.m. 
and  p.m.  pullout. 

Milestones: 

FY2001  FY2002 

96.5%  97%  (%  of  scheduled  service  hours  delivered} 

From  Mum  Seoxe  Slandordi  Report.  FY200I  and  FY2002 
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Changes  in  Measurement  Since  the  Audit  Period 

Subsequent  to  the  audit  period,  Muni  adjusted  the  milestones  for 
scheduled  service  liours.  The  FY2003  and  FY2004  milestones 
were: 

• FY2003  - 97.5% 

• FY200^-98.5% 

Action  Plan 

Develop  a Plan  (or  Maximi/ing  Service  Provision 
During  "Low  Availability  Periods" 

The  milestone  taiy,cts  adopted  liy  MTA  for  FY200I  and  FY2002  for 
this  service  standard  were  consistent  with  achieving  the  FY2004 
attainment  of'No  loss  than  98.5%  of  service  hours  delivered  on  July  I , 
2004."  The  [Ji  ogross  between  FY200 1 of  94.53%  systemwide  service 
hours  delivered  to  FY2002  of  97.80%  of  systemwide  service  hours 
dolivei  od  was  significant.  However,  unaudited  fourth  quarter  results 
from  FY2003  do  not  indicate  that  progress  is  continuing,  since  only 
94.52%  of  scheduled  service  hours  were  delivered  systemwide. 

It  is  realistic  to  assume  that  in  the  future  MTA  will  experience 
various  "spikes"  In  o[)crator  availability  due  to  changes  in  the 
rates  of  Claiming  Industrial  and  Claiming  Industrial  for  Assault  (i.e.. 
operator  alisences  due  to  workers  compensation)  and  fluctuations 
in  the  hiring  process  due  to  potential  cuts  in  support  services.  It 
would  be  prudent  that  Operations  Managers  draft  a plan  to  consider 
reallocation  of  operator  resources  between  modes  in  order  to 
equalize  any  missed  service  systemwide.  It  is  proLubIc  that  specific 
shortages  in  operator  availability  in  any  mode  will  not  be  addressed 
in  a systemwide  target  but  nonetheless  should  be  minimized  on  a 
mode  and  division  liasis.  Notwithstanding  labor  agreements  and 
past  practices.  Operation  Managers  should  take  all  necessary  steps 
to  maintain  the  highest  levels  of  on-scrcot  service  provision  when 
there  are  spikes  in  operator  availability. 


Use  of  Cumulative,  Annual  Performance  vs.  Quarterly  Performance 

Rather  than  using  fourth  quarter  results  to  determine  achievement 
of  annual  goals,  the  auditor  recommends  that  cumulative  or  average 
annual  performance  be  used  instead.  There  are  a number  of  internal 
and  external  factors  that  lead  to  substantial  variation  in  performance 
between  quarters.  Reporting  year-to-date  or  cumulative  annual 
performance  may  provide  a better  picture  of  systemwide  and  modal 
performance  over  the  entire  year.  Quarterly  performance  could 
still  be  used  to  determine  variations  in  progress  toward  meeting 
performance  goals  throughout  the  year 


Proposition  E Municipal  Transportation  Quality  Review 

DRAFT  Final  Report 


3a.  Missed  service  due  to  either  insufficient  vehicles  or  driver 
unavailability  as  a percentage  of  scheduled  service  hours. 


Goal: 

No  less  than  1.5%  on  July  1,  2004  

rurpose: 

To  measure  service  hours  through  available  operators  and  available  o.iuiumont 

Defmition  of 
Measurement: 

eTuLmLranH"r'"',“'  '"r  oporalors  ami  ' 

equipment  and  percentage  of  equipment  available  dailv 

Method  of 
Measurement: 

uih  uperaiors  ano  equipment  are  measured  as  to  the  total  number  of  hours  in  service  missod  as  n 
- 'he  total  scheduled  hours.  Data  comes  from  tho  online  Dispatching  Systom. 

Milestones: 

FY2001  FY2002  — 

3.5%  3%  (%  of  scheduled  service  hours  delivered) 

From  Muni  Serv<e  Sujndordt  Report,  FY200I  and  fY2002  
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Reported  Performance 

This  service  standard  is  the  inverse  of  Service  Standard  2a.  Rather 
than  the  percent  of  scheduled  service  hours  that  are  delivered,  this 
service  standard  reports  the  percent  of  scheduled  service  hours  that 
are  misserl  as  a result  of  insufficient  vehicles  or  driver  unavailability. 

At  the  end  of  FY200 1 , Muni  was  unable  to  meet  Its  goal  of  no  more 
than  3.5%  of  scheduled  service  hours  missed,  cither  systemwide  or 
for  any  particular  mode  or  division,  except  for  the  Kirkland  motor 
coach  division.  Missed  hours  were  due  primarily  to  unavailability  of 


operators,  rather  than  equipment.  There  was  significant  improvement 
in  FY2002.  however,  and  the  goal  of  no  more  than  3%  of  scheduled 
service  hours  missed  was  met  for  most  divisions  (except  cable  car) 
and  systemwide. 

Any  improvement  gains  appear  to  have  been  lost  in  FY2003,  since 
the  goal  was  not  met  by  any  of  the  divisions  or  systemwide.  First 
quarter  results  for  FY2004  do  not  indicate  much  improvement. 
Table  A-5  summarizes  the  fourth  quarter  for  fiscal  years  2001, 2002 
and  2003  and  the  first  quarter  results  for  FY2004. 


Table  A-5  Porcontago  of  Missed  Service,  FY200 1 - FY2004 


4th  Quartor  FY2001 

4th  Quarter  FY2002 

4th  Quarter  FY2003 

1st  Quarter  FY2004 

FY2001  Qoal;  3.5% 

FY2002  Goal:  3% 

FY2003  Goal:  2.5% 

FY2004  Goal:  1.5% 

%Hrs 

% Mrs 

% Hrs 

%Hrs 

%llr.s 

MIssmt 

% of  fiital 

% Mrs 

Missod 

% of  Total 

%Hrs 

Missed 

% of  Total 

% Hrs 

Missed 

% of  Total 

Missiid  linn 

dim  to 

Schodtilnd 

MissmI  dun 

1 duo  to 

,,  Schodiilod 

Missed  duo 

duo  to 

Scheduled 

Missed  due 

; due  to 

Scheduled 

to  niKiriitiir 

r(|iil|)innnt 

Hours 

to  0|inratnr 

C()uipmnnt  ‘ 

Hours 

to  Oporntor 

Equipment 

Hours 

to  Operator* 

, Equipment 

Hours 

Unnviillnhln 

Uniiviilliilili) 

MissmI 

Unnvaiinhin 

Unnuailnhln 

Missod 

Unnvnilnhlo 

Unavnilablo 

Missed 

Unavailable 

Unavailable 

Missed 

Wlolnr  ('nnrii 

b.Ul% 

0.74% 

5.34% 

1.90% 

0.05% 

1.95% 

5.42% 

0.03% 

5.46% 

3.98% 

0.15% 

4.12% 

Flynn 

(i.77% 

0.79% 

7.50% 

l.G9% 

0.05% 

1.74% 

6.59% 

0.08% 

6.66% 

5.79% 

0.15% 

5.94% 

Woods 

l).3d% 

0,U1% 

5.44% 

2.30% 

0.09% 

2.38% 

6.45% 

0.01% 

6.45% 

4.47% 

0.20% 

4.67% 

Kirklnml 

XM]% 

0.00% 

3.40% 

1.50% 

0.01% 

1.51% 

3.19% 

0.03% 

3.23% 

1.91% 

0.07% 

1.99% 

Irnllnv 

bxm 

0.10% 

5.19% 

1.40% 

0.39% 

1.85% 

2.40% 

0.12% 

2.52% 

2.20% 

0.09%  ' 

2.29% 

Potriirii 

b.m 

0.1  B% 

5.20% 

2.19% 

0.07^. 

2.26% 

f.97% 

0.15% 

2.13% 

3.23% 

0.17% 

3.39% 

Prosidio 

b.lH% 

0,01% 

5.10% 

0.57% 

0.77% 

1.34% 

2.92% 

0.09% 

3.01% 

1.06% 

0.01% 

1.07% 

LIW 

T'/0% 

1.22% 

5.91% 

1.75% 

0.1  r 

1.86% 

5.48% 

0.021% 

5.69% 

3.30% 

0.48% 

3.78% 

Cnlilo  ['(11 

;.bb% 

0.05% 

7.00% 

7.93% 

0.25% 

8.18% 

26.38% 

0.30% 

26.68% 

9.72% 

0.15% 

9.87% 

Systoinwido 

5.15% 

0.3  r;. 

5.47% 

7.02% 

0.18% 

2.20% 

5.37H 

0.10% 

5.48% 

3.54% 

0.19% 

3.73% 

Fi-oi^i  Muni  SwvKt*  fY^OOl,  fV^OO^.  fY200i  otyd  l\t  Quottn  FYiCKH 

In  ItAllc*  not  hren  auiIIkhI, 
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g Proposition  E Municipal  Transportation  Quality  Review 

■ Verification  of  Definition  and  Method  of 

■ Measurement  During  the  Audit  Period 

I During  the  audit  period,  data  were  entered  on  a daily  basis  into 

I the  Automated  Dispatch  system  by  the  division  dispatchers.  The 

I dispatchers  followed  regularized  procedures  for  showing  runs 
without  a driver  assignment  and/or  without  equipment  available. 
Division  Managers  were  responsible  for  reviewing  automated 
dispatch  data. 

The  Automated  Dispatch  software  provided  a daily  aggregation 
of  data  on  operator  and  vehicle  availability.  This  information  was 
tabulated  to  provide  monthly  and  quarterly  assessments.  The 
percentage  availability  was  calculated  using  data  from  the  Automated 
Dispatch  software  and  scheduled  service  information  from  the 
Schedule  Department. 

Verification  of  Reported  Performance 
During  the  Audit  Period 

j The  responsible  position  for  performing  this  function  was  eliminated 
I in  the  current  FY2003-2004  budget;  the  function  is  currently 
I performed  by  a manager  in  the  scheduling  section  of  the  Operations 
Division.  Procedural  documentation  for  Proposition  E reporting  for 
FY200I  and  FY2002  was  not  available.  Audit  verification  of  initial 
source  data  was  accomplished  by  query  of  the  Auto  Dispatch  system 
and  sggregation/summarization  was  accomplished  by  manual  means 
utilizing  hard  copies  of  weekly  missed  service  reports.  Based  on  this 
information,  the  percent  availability  of  operators  and  equipment  for 
FY2002  appears  to  have  been  reported  correctly. 
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Problems  or  Issues  Identified  During  the  Audit  Period 

Other  than  difficulties  encountered  due  to  staff  changes,  no  other 
problems  or  issues  were  identified.  Although  performance  goals 
were  achieved  in  FY2002,  future  performance  achievement  appears 
CO  be  problematic. 

Changes  in  Measurement  Since  the  Audit  Period 

Subsequent  to  the  audit  period.  Muni  adjusted  the  milestones  for 
missed  scheduled  service  hours.  The  FY2003  and  FY2004  milestones 
were: 

• FY2003  -2.5% 

• FY2004  -I.5% 

Auditor  Recommendations 

This  service  standard  should  be  eliminated  since  it  Is  simply  the 
inverse  of  Service  Standard  2a.  The  Citizen  s Advisory  Committee 
(CAC)  has  also  recommended  that  this  Service  Standard  be  removed, 
since  it  is  a duplicate  of  Standard  2a. 
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11"  to  pTclfup 

3 minutes  or  less  1 another  vehicle  on  the  same  route  with  space  for  all 
waiting  passengers. 


GonI: 

Loss  than  f)%  — 

Piirposo: 

Tn  mnasiirn  crowdinn  in  voliicles.  — 

Dofinition  of 
Mcnsiiromont: 

rY2001:  Monthly  nicasiircmont  of  the  5 most  patronized  routes  for  each  division  Such  checks  shall  be 
no  less  often  than  10  weekdays  per  month  on  5 different  routes  per  month  in  each  division  beginning  with  the 
most  patronized  route,  progressing  in  the  order  of  declining  route  patronage,  and  ending  v^nth  the  least  patronized 
roiito.  The  cycle  shall  then  be  repeated.FY2002:  Each  line  will  be  checked  twice  a year  Such  checks  sha 
ho  conducted  no  less  often  than  10  weekdays  and  weekends  per  check.  An  annual  checking  schedule  shall  be 
ostalilished,  with  the  lines  to  he  checked  each  month  randomly  selected.  To  the  extent  automated  systems  can  be 
siibstitiitod  at  less  cost  for  such  checks,  or  the  measurement  of  any  performance  standard,  such  systems  must  be 

used.  ^ ^ tt;: — 

Mnthnil  of 
Monsmnmont: 

Chock  the  (losignatod  linos  using  criteria  of  -1  minute  to  .4  minutes.  "Periods  of  time  includes  morning  rush  (fa 
n.ni.  to  9 a.m.)  mitlday  (9  a.m.  • 4 p.ni.)  evening  rush  (4  p.m.  to  7 p.m.)  and  night  (7p.m.  to  1 a.m.).  Supervisory 
personnel  shall  conduct  a one-hour,  on-time,  and  load  standard  check  at  a...  FY2001:  ...maximum  load  point  at 
mid-rnuto  dining  all  four  time  periods  stated  above.FY2002:  ...maximum  load  point  at  mid-route  during  all  four 
limo  periods  stated  above.  Pass-ups  shall  be  checked  during  such  times  at  one  stop  before  the  maximum  loa 

point  — 

Milostonns: 

rY20Ql  FY2002 

Loss  than  5%  loss  than  9% 

I ivm  Aliinl  Spivkr  Standoii/i  Rp|'o»t.  fYiOOf  ond  fy}002 
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Reported  Performance 

In  FY200 1 .fewer  than  0.35%  of  vehicles  sampled  passed  up  passengers 
due  to  overcrowding  without  being  followed  by  another  vehicle  able 
to  accommodate  all  passengers  within  three  minutes.  In  the  fourth 
quarter  of  FY200 1 , the  pass  up  percentage  was  zero.  In  FY2002,  the 
highest  pass-up  percentage  was  0.71%  and  the  lowest  was  0%.  Thus, 
in  both  FY200 1 and  FY2002,  the  goal  of  lower  than  a 5%  pass-up  rate 
was  not  only  achieved  but  surpassed.  The  highest  for  any  of  the  lines 


sampled  was  5.56%  for  the  9AX  San  Bruno  (November  2000)  and 
5.88%  for  the  I AX  California  A Express  (July  2001). 

The  percentage  of  pass-ups  spiked  during  several  months  in  FY2003, 
and  were  relatively  high  for  the  M-Oceanview  and  the  N-judah 
during  the  first  quarter  of  FY2004.  Table  A-6  presents  systemwide 
pass-up  results  for  fiscal  years  200 1 . 2002,  and  2003  and  for  the  lines 
measured  during  the  first  quarter  of  FY2004. 


Table  A-6  Percent  of  Pass  Ups  (from  checks  at  MLP  for  the  five  most  patronized  linos),  FY200 1 - FY2004 


FY2001 

Average 

FY2002 

Average 

FY2003 

Average 

\ 

FY2004  Avorngu 

Month 

Systemwide 

Systemwide 

Systemwide 

M- 

Oceanview 

N Judah 

30-Siuckton 

38L  Goary 
Llmltud 

44 

O'Shnughnossy  | 

Systninwlilo 

July 

0.15% 

0.04% 

0.72% 

4.35% 

4.92% 

0.00% 

0.00% 

0.00% 

L71% 

August 

0.02% 

0.31% 

0.00% 

8.82% 

5.66% 

0.00% 

0.00% 

0,00% 

2.90% 

September 

0.05% 

0.45% 

5.49% 

3.77% 

W.94% 

4.92% 

0.00% 

0.00% 

4.41% 

October 

0.04% 

0.13% 

6.78% 

November 

0.09% 

0% 

8.57% 

December 

0.35% 

0.09% 

6.09% 

January 

0.03% 

0.34% 

0.53% 

February 

0.04% 

0.24% 

0.00% 

March 

0.10% 

0.10% 

0.00% 

April 

0% 

0.24% 

0.00% 

May 

0% 

0.27% 

4.82% 

June 

0% 

0.71% 

0.00% 

From  Mom  Service  Stoodorrfi  Report. Qoorter  fY200/.fY2002.FY2003  and  lit  Qoorfer  FY2004 
Data  m italics  have  noc  been  audited. 
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Verification  of  Defin/t/o.,  and  Method  of  Meatureatent  During  the  Audi.  Period 


FY200I 

The  following  changes  in  sampling  methodology  were  approved  by 
the  MTA  Board  and  subsequently  implemented  in  FY200I. 

• Light  rail  (LRV)  and  cable  car  divisions  were  combined  into 
one  rail  sampling  rotation  with  results  reported  separately: 
ciglit  cliecks  per  montli  were  reduced  to  five  or  four  checks 
per  month. 

• Three  motorconch  divisions  were  combined  into  one  motor- 
coacli  sam()ling  rotation;  1 5 checks  per  month  were  reduced 
to  five  checks  per  month. 

• Two  trollcycoach  divisions  were  combined  into  one  trolley- 
coacli  sampling  rotation;  10  cliecks  per  month  wore  reduced 
to  five  chocks  per  month. 

Linos  wore  sampled  witliin  cacli  mode  according  to  ranked 
patronage  as  described  in  tlio  definition  of  measurement.  At  least 
ten  weekday  cliecks  were  performed  during  eacli  montli  for  motoi 
coacli  and  trolley  coacli  modes.  Liglit  rail  and  cable  car  modes  wore 
in  compliance  with  tlio  definition  of  measurement  for  the  months 


they  were  sampled.  All  time  periods  were  sampled  in  accordance 
with  the  definition  and  method  of  measurement. 

fY2002 

The  selection  of  routes  was  coordinated  with  the  Data  Collection 
Plan  (DCP)  adopted  by  Muni  for  provision  of  data  for  internal  planning 
purposes  and  to  satisfy  the  annual  NTD  (and  previously  UMTAy 
FTA  Section  15)  ridership  reporting.  The  FY200I  methodology  for 
sampling  15  lines/checks  per  month  was  modified  for  FY2002  to 
include  nine  motorcoach  lines,  three  to  four  trolleycoach  lines,  and 
two  to  three  rail  lines.  A random  selection  process  replaced  the 
ranked  ridership  rotation  process.  Specific  time  periods  conformed 
to  the  DCP  and  included,  as  a minimum,  the  time  periods  defined  in 

Proposition  E. 


Proposition  E Municipal  Transportation  Quality  Review 


Verification  of  Reported  Performance 
During  the  Audit  Period 

An  audit  of  FY200I  and  FY2002  data  confirmed  that  pass  up  data 
was  reported  correctly  for  all  but  the  fourth  quarter  of  FY2001. 
For  the  fourth  quarter  of  FY200I,  zero  pass  ups  were  reported  in 
the  Service  Standards  Report,  however  source  data  showed  that 
five  pass  ups  occurred  during  that  quarter.  It  is  likely  that  updates 
or  changes  to  data  were  not  transmitted  to  the  Superintendent  of 
Performance  Monitoring  and  Analysis. 

For  the  second,  third  and  fourth  quarters  of  FY200I,  the  number 
of  checks  at  the  maximum  load  point  reported  in  the  4th  Quarter 
FY200I  Service  Standards  Report  were  incorrect.  Following  are  the 
correct  values: 


FY2001 

2nd  Quarter 

3rd  Quarter 

4th  Quarter 

Number  of  Checks  at  MLR 

8,881 

7,373 

9,633 

These  reporting  problems  were  considered  minor  and  correctable. 


Problems  or  Issues  Identified  During  the  Audit  Period 
No  additional  issues  or  problems  were  identified. 

Chonges  in  Measurement  Since  the  Audit  Period 

Subsequent  to  the  audit  period.  Muni  adjusted  the  definition  of 
measurement  for  pass-ups,  as  follows: 

• Pass-ups  shall  be  conducted  no  less  often  than  10 
weekdays  per  month.  At  the  beginning  of  each  quarter, 
supervisory  personnel  will  review  all  the  lines  checked  in  the 
previous  quarter,  and  identify  the  five  lines  with  the  highest 
load  factors,  and  the  time  period  these  load  factors  occurred. 
Supervisory  personnel  will  then  check  those  five  lines,  during 
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the  time  period  that  the  high  load  factor  occurred,  each  month 
during  the  coming  quarter  for  pass-ups.  Supei  visory  person- 
nel will  check  to  see  if  any  PSRs  for  pass-ups  were  made  for 
any  of  those  five  lines,  and  if  the  location  of  the  pass-up  was 
recorded.  If  it  was  recorded,  supervisory  personnel  will  use 
that  point  as  the  point  to  check  for  pass-ups.  If  there  arc  no 
locations  recorded,  supervisory  personnel  will  use  the  stop 
just  before  the  MLR 

Aud/tor  Recommendations 

Performance  results  have  been  reported  in  two  different  formats. 
The  4th  Quarter  FY2001  Service  Standards  Report  contained 
a system-wide  table  summarized  by  quarter  which  Included  the 
following  information:  number  of  pass-ups,  percentage  of  pass-ups, 
number  of  checks  at  MLR,  and  the  lines  with  more  than  two  pass-ups. 
The  4th  Quarter  FY2002  Service  Standards  Report  did  not  contain 
a cumulative  (each  quarter  and  annual)  systemwide  table.  Because 
the  sampling  plan  for  lincs/modcs  varies  by  month,  both  a c|uartcrly 
and  an  annual  performance  summary  should  be  presented. 

Two  changes  are  recommended  In  reporting: 

• Include  a systemwide  pass-up  table  by  quarter  witli  both 
number  and  %;show  previous  quarters  and  present  an  annual 
summary;  and 

• Track  individual  pass-up  line  pciformance  over  each  c|uartcr 
of  each  fiscal  year 

The  auditor  also  recommends  that  Service  Standard  4a  be 
coordinated  with  Service  Standard  5a  so  that  all  lines  identified  as 
exceeding  85%  of  capacity  during  peak  periods  are  monitored  as 
part  of  Service  Standard  4a.  Additional  data  available  through  the 
Passenger  Service  Reports  database  should  be  utilized  to  conduct 
follow-up  traffic  checks  on  lines  chronically  identified  as  having  pass- 
ups  or  peak  period  loads  that  exceed  85%  of  capacity. 


PagoA-23 


San  Francisco  Municipal  Railway 


5a.  Peak  period  passenger  load  factors. 


Goal: 

No  (ireatcr  than  85%  of  combined  seating  & standing  capacity^^^^ 

Piirposo: 

To  moasuro  load  factors  at  poak  periods  

Dofinition  of 
Moasurcmcnt: 

Monthly  measuremoni  of  the  5 most  patronized  routes  for  each  division.  Such  checks  shall  he  conducted  no 
less  often  than  1 0 weekdays  per  month  on  5 different  routes  per  month  in  each  division,  beginning  with  the 
most  patronized  route,  progressing  in  the  order  of  declining  route  patronage,  and  ending  with  the  least  patronized 
route.  The  cycle  shall  then  he  repoated.FY2002;  Each  line  will  be  checked  twice  a year.  Such  checks  shall 
ho  conducted  no  loss  often  than  ID  weekdays  and  weekends  per  check.  An  annual  checking  schedule  shall  be 
established  lor  the  routes.  The  order  in  which  the  routes  are  checked  will  he  determined  monthly  through  a 
random  soloction  process.  To  the  extent  automated  systems  can  he  substituted  at  less  cost  for  such  checks,  or 
the  moasuromont  of  any  performance  standard,  such  systems  must  be  used. 

Mothnil  of 
Monsiirumont: 

Method  of  Moasuromont:  " Periods  of  time  includes  morning  rush  (6  a.m.  To  9 a.m.)  midday  (9  a.m.-  4 p.m.) 
ovening  rush  |4  p.m.  To  7 p.m.)  and  night  (7  p.m.  To  1 a.m.).  Supervisory  personnel  shall  conduct  a one-hour,  on 
time  and  load  standard  chock  at  a maximum  load  point  at  mid  route  during  all  four  time  periods  stated  above. 

Milostnnns: 

IY2()()1  FY20Q2 

No  Groatur  than  i!5%  No  gieater  than  85% 

r-i'on\  Muni  Sff vlt  p Sf(»iil(r»i/»  Rp|>of  f.  fYJOO  I and  fY2002 
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Reported  Performance 

Muni  was  unable  to  meet  its  goal  of  no  lines  over  85%  capacity 
in  either  FY200I  or  FY2002.  Due  to  changes  in  the  sampling 
methodology  between  those  two  years,  annual  performance  results 
could  not  be  accurately  compared.  This  is  discussed  further  in  the 
section  "Problems  or  Issues  Identified."  Although  the  goal  was  not 
met  in  FY2003,  substantial  improvement  was  made,  with  only  10 
lines  over  85%  capacity.  The  following  table  shows  the  number  of 
lines  that  failed  to  meet  the  standard  of  no  greater  than  85%  of 
combined  seating  and  standing  capacity  in  fiscal  years  2001,  2002, 
2003  and  the  first  quarter  of  FY2004. 


Table  A-7  Number  of  Lines  that  Exceeded  85%  Capacity, 
FY200I  -FY2004 


Month 

FY2001 

FY2002 

FY2003 

FY2004 

July 

8 

3 

0 

0 

August 

4 

5 

4 

2 

September 

5 

2 

0 

0 

October 

2 

4 

/ 

November 

4 

0 

0 

December 

1 

0 

0 

January 

2 

1 

/ 

February 

3 

0 

2 

March 

6 

1 

0 

April 

6 

4 

/ 

May 

3 

2 

/ 

June 

4 

2 

0 

TOTAL 

48 

24 

10 

Fpo^  AV«  SenrK*  Standanh  fUpoa.4th  Quow  FY200I.  FY2002.  FY2003  and  /«  Qu»i*f  m004 
Dati  m it2l<s  noc  boen  audttad 
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Verificotion  of  Definition  and  Alethod  of 
Aieasurement  During  the  Audit  Period 

mm 

The  following  changes  in  sampling  methodology  wore  approved  by 
the  MTA  Board  and  subsequently  implemented  in  FY200I: 

• Light  rail  (LRV)  and  cable  car  divisions  were  combined  into 
one  rail  sampling  rotation  with  results  reported  separately: 
eight  checks  per  month  wore  reduced  to  five  or  four  chocks 
per  month. 

• Three  motorcoach  divisions  were  combined  Into  one  motor- 
coach  sampling  rotation;  1 5 checks  per  month  were  reduced 
to  five  checks  per  month. 

• Two  trolleycoach  divisions  were  combined  Into  one  trolley- 
coach  sampling  rotation;  10  chocks  per  month  were  reduced 
to  five  checks  per  month. 

Lines  were  sampled  within  each  mode  according  to  ranked 
patronage  as  described  in  tfio  definition  of  measurement,  At  least 
ten  weekday  checks  were  performed  during  cacli  month  for  motor 
coach  and  trolley  coach  modes.  Light  rail  and  cable  car  modes  were 
in  compliance  with  the  definition  of  measurement  for  tlio  months 
they  were  sampled.  All  time  periods  wore  sam()lcd  In  accordance 
with  the  definition  and  method  of  measurement. 

fY2002 

The  selection  of  routes  was  coordinated  with  tlic  Data  Collection 
Plan  (DCP), adopted  by  Muni  for  provision  of  daui  for  Internal  planning 
purposes  and  to  satisfy  the  annual  NTD  (and  previously  UMTAy 
FTA  Section  15)  ridership  reporting.  The  FY200I  methodology 
for  sampling  1 5 lines/chccks  per  month  was  modified  for  FY2002 
to  include  nine  motorcoach  lines,  three  to  four  trolleycoach  lines, 
and  two  to  three  rail  lines.  A random  selection  process  replaced 
the  ranked  ndcrship  rotation  process.  Specific  locations  and  time 
periods  conformed  to  the  DCP  and  included,  as  a minimum,  the 
periods  defined  in  Proposition  E.  p3ooA-25 
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Verification  of  Reported  Performance 
During  the  Audit  Period 

Based  on  an  audit  of  data  for  FY200I  and  FY2002,  reported  results 
iTiJitcii  those  subiTiittcd  by  Muni  data  staff. 

Problems  or  Issues  Identified  During  the  Audit  Period 

Althougli  the  total  number  of  lines  checked  was  about  the  same  for 
both  FY200I  and  FY2002,  tlie  number  of  lines  checked  within  each 
mode  changed  lietwecn  the  two  years.  In  FY200 1 , three  to  five  lines 
were  cliecked  monthly  in  eacli  of  the  tiuee  modes  - rail,  trolley  and 
motor  coach.  In  FY2002,  monthly  cliecks  consisted  of  one  to  three 
rail  lines,  throe  to  four  trolley  lines,  and  nine  to  ten  motor  coach 
linos.  While  tills  cliaiiRe  led  to  a better  bal.inco  of  sampling  between 
modes,  it  made  it  impossible  to  accurately  compare  performance 
l)otwocn  FY200I  and  FY2002. 

In  FY200I,'10  out  of  the  approximately  175  linos  checked, or  27.4%, 
had  load  f.ictors  that  exceeded  85%,  In  FY2002,  23  out  of  the 
approximately  176  lines  chocked,  or  13.1%,  had  load  f.ictors  that 
exceeded  85%,  Analysis  of  the  24  lines  that  improved  load  factors 
lictwoen  FY200I  and  FY2002  showed  that  there  was  a reduction 
from  46  monthly  samples  above  85%,  to  only  six  monthly  samples 
above  85%,  For  those  same  24  lines,  the  rate  of  sampling  was 
reduced  by  30,1%  between  FY200I  and  FY2002  from  83  samples  to 
58  s.implos. 

Changes  in  Alcasurcmcnt  Since  the  Audit  Period 

Subsequont  to  the  audit  poriod.  Muni  has  made  no  changes  to 
the  liotinition  of  moasuremont,  method  of  measurement,  nor  the 
milestones. 


Auditor  Recommendations 

In  all  cases  of  non-compliant  lines,  sampling  frequency  should  be 
increased  from  the  previous  year  until  a trend  in  compliance  is 
noted.  This  will  enable  an  accurate  evaluation  of  improvement  in 
line  performance  from  year  to  year. 

A more  meaningful  method  of  reporting  may  be  to  capture  individual 
line  load  factors  by  1 5-minute  periods  (quarter-hour  peak  periods), 
rather  than  for  a full  hour.  This  coincides  with  schedule  evaluation 
and  revision  analysis  processes,  and  would  enable  Muni  staff  to 
better  pinpoint  time  periods  that  require  greater  capacity  (in  order 
to  relieve  overcrowding). 

Performance  results  have  been  reported  in  two  different  formats. 
The  4th  Quarter  FY200I  Service  Standards  Report  contained 
a system-wide  table  summarized  by  quarter  which  included  the 
following  information;  number  of  pass-ups,  percentage  of  pass-ups, 
number  of  checks  at  MLP,and  the  lines  with  more  than  two  pass-ups. 
The  4th  Quarter  FY2002  Service  Standards  Report  did  not  contain 
a cumulative  (each  quarter  and  annual)  systemwide  table.  Because 
the  sampling  plan  for  lines/modes  varies  by  month,  both  a quarterly 
and  an  annual  performance  summary  should  be  presented. 

The  auditor  also  recommends  that  Service  Standard  5a  be 
coordinated  with  Service  Standard  4a  so  that  all  lines  identified  as 
exceeding  85%  of  capacity  during  peak  periods  are  monitored  as 
part  of  Service  Standard  4a.  Additional  data  available  through  the 
P.isscngor  Service  Reports  database  should  be  utilized  to  conduct 
follow-up  tiMffic  checks  on  lines  chronically  identified  as  having  pass- 
ups  or  peak  period  loads  that  exceed  85%  of  capacity. 
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6a.  Actual  headways  against  scheduled  headways  on  all  radial,  express, 
cross-town,  secondary,  and  feeder  lines  for  peak,  base,  evening,  and 
late  night  services. 


Goal: 

Operate  no  less  than  80%  of  the  time  within  30%  or  10  minutes  ol  schoilulod  headway  (whichovor  is  loss). 
FY2002:  Operate  no  less  than  85%  of  the  time  within  30%  or  10  minutes  of  scheduled  headway  (whichovor  is 
less). 

Purpose: 

Measurement  of  scheduled  headways  against  actual  headways. 

Definition  of 
Measurement: 

FY200 1 : Will  measure  average  headway  monthly  and  will  designate  typo  ol  lino  as  part  ol  Sorvico  Standard  1 a 
- % of  vehicles  that  run  on  time  according  to  published  schodulos.  Muasuremont  of  tho  5 most  patronized  routns 
for  each  division.  Such  checks  shall  be  conducted  no  less  often  than  10  weekdays  and  wookends  pur  month 
on  5 different  routes  per  month  in  each  division,  beginning  with  the  most  patronized  route,  progressing  in  the 
order  of  declining  route  patronage,  and  ending  with  the  least  patronized  route.  Tho  cycio  shall  then  bo  repeated. 
FY2002:  Each  line  will  be  checked  twice  a year.  Such  checks  shall  bo  conducted  no  less  often  than  ID  weekdays 
and  weekends  per  check.  An  annual  checking  schedule  shall  be  established  for  tho  rentes.  Tho  order  in  which 
the  routes  are  checked  will  be  determined  monthly  through  a random  selection  process.  To  tho  extent  automated 
systems  can  be  substituted  at  less  cost  for  such  checks,  or  the  rnoasuremont  of  any  performance  standard,  such 
systems  must  be  used. 

Method  of 
Measurement: 

Method  of  Measurement:  Check  the  headways  of  designated  linos.  ‘'Periods  of  time  includes  morning  rush  (6 
a.m.  to  9 a.m.)  midday  (9  a.m.  ■ 4 p.m.l  evening  rush  (4  p.m.  to  7 p.m.)  and  night  |7p.m.  to  1 a.m.).  Supervisory 
personnel  shall  conduct  a one-hour,  on-time,  and  load  standard  chock  at  a maximum  load  point  at  mid-route  during 
all  four  time  periods  stated  above." 

Milestones: 

FY2001  FY2002 

80%  85% 

From  Murw  Stmct  Stondanh  Report,  FY200I  end  Ff2002 
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Reported  Performance 

For  the  fourth  quarter  of  FY200 1,  systemwide  headway  adherence 
was  56.8%  (the  percentage  of  sampled  headways  that  were  wit  in 
30%  or  10  minutes  of  the  scheduled  headway,  whichever  was  less). 
In  the  fourth  quarter  of  FY2002,  systemwide  headway  adherence 
rose  to  72.1%,  liowcver  results  between  FY200I  and  FY2002  are 
not  directly  comparaljle  due  to  a change  in  sampling  methodology 
between  FY200I  and  FY2002.  Headway  adherence  did  not  improve 
in  FY2003  or  the  first  quarter  of  FY2004.  Table  A-8  shows  the 
average  percentage  of  headway  adherence  for  the  lines  measured 
withii;  each  month  of  fiscal  years  2001,  2002,  2003  and  the  first 
quarter  of  FY2004. 


Table  A-0  Average  Percentage  of  Lines  Adhering  to 
Scheduled  Hcadw.ays,  FY200 1 - FY2004 


Month 

FY?0ni  j 

FY20fl2 

FY2003 

FY2004 

July 

04.0% 

73.3% 

79.0% 

Aii(|iist 

44.4% 

00.5% 

65.8% 

68,2% 

Soptmnhm 

Gl).(i% 

74.7% 

05.8% 

67.8% 

Gctolioi 

4G.I)% 

59.9% 

77.7% 

Novomluu 

r)4.o% 

79.8% 

70.2% 

Gnnniiihnr 

50.2% 

71,3% 

59.4% 

Jnmmry 

54.5% 

05.0% 

02.7% 

Fohriimy 

43,7% 

07.1% 

69.7% 

March 

50.5% 

7G.1% 

66.7% 

Apiil 

04.3% 

00,3% 

81.1% 

May 

53.4% 

73.3% 

67,0% 

Jiinn 

57.4% 

tUl.4% 

75.9% 

F,xw>  M.ml  Sfivl,o  SloiK(,ml>  Rrpoa^it.  OiMMr.  m00l.m002.  FV003  onJ  l.i  fYiOW 
D.M.1  In  UaIIcs  Ilwc  noi  bren  .muIikhI. 


Figure  A- 1 Headway  Adherence 
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Verification  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

mool 

The  following  changes  in  sampling  methodology  were  approved  by 
the  MTA  Board  and  subsequently  implemented  in  FY200I; 

• Light  rail  (LRV)  and  cable  car  divisions  were  combined  into 
one  rail  sampling  rotation  with  results  reported  separately; 
eight  checks  per  month  were  reduced  to  five  or  four  checks 
per  month. 

• Three  motorcoach  divisions  were  combined  into  one  motor- 
coach  sampling  rotation:  1 5 checks  per  month  were  reduced 
to  five  checks  per  month. 
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• Two  trolleycoach  divisions  were  combined  into  one  trolley- 
coach  sampling  rotation:  10  checks  per  month  were  reduced 
to  five  checks  per  month. 

Lines  were  sampled  within  each  mode  according  to  ranked 
patronage  as  described  in  the  definition  of  measurement.  At  least 
ten  weekday  checks  were  performed  during  each  month  for  motor 
coach  and  trolley  coach  modes.  Light  rail  and  cable  car  modes  were 
in  compliance  with  the  definition  of  measurement  for  the  months 
they  were  sampled.  All  time  periods  were  sampled  in  accordance 
with  the  definition  and  method  of  measurement. 

fY2002 

The  selection  of  routes  was  coordinated  with  the  Data  Collection 
Plan  (DCP), adopted  by  Muni  for  provision  of  data  for  internal  planning 
purposes  and  to  satisfy  the  annual  NTD  (and  previously  UMTA/ 
FTA  Section  15)  ridership  reporting.  The  FY200I  methodology  for 
sampling  15  lines/checks  per  month  was  modified  for  FY2002  to 
Include  nine  motorcoach  lines  and  three  to  four  trolleycoach  lines. 
A random  selection  process  replaced  the  ranked  ridership  rotation 
process.  Specific  locations  and  time  periods  conformed  to  the  DCP 
and  included,  as  a minimum,  the  Proposition  E defined  time  periods. 

The  two  rail  modes  (light  rail  and  cable  car)  as  well  as  major  radial 
lines  were  removed  from  the  measurement  for  FY2002.  The 
rationale  for  this  decision  was  that  the  standard  of  30%  or  better 
headway  adherence  was  not  applicable  to  smaller  headways  because 
it  would  not  be  meaningful.  Overall,  60  lines  were  sampled  in  FY2002, 
compared  with  82  lines  in  FY200I.  (This  change  was  approved  by 
the  MTA  Board.)  Measurement  of  headway  adherence  for  all  modes 
Was  resumed  in  FY2004. 
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Venficotion  of  Reported  Performance 
During  the  Audit  Period 

Based  on  an  audit  of  data  for  FY200I  and  FY2002,  reported  results 
match  those  submitted  by  Muni  data  staff. 

Problems  or  Issues  Identified  During  the  Audit  Period 

Muni  has  been  unable  to  meet  Its  headway  adherence  goals  during 
all  years  of  measurement.  Adjustments  to  the  method  and  definition 
of  measurement  had  some  impact  on  performance  results,  however 
changes  that  were  made  In  FY2002  and  FY2003  were  reversed  in 
FY2004.  Consequently,  it  is  necessary  to  further  examine  the  reasons 
for  failure  to  meet  performance  goals  as  well  as  the  goals  themselves, 
the  definition  of  measurement,  and  the  method  of  measurement. 

Chonges  in  Alcasuremcnt  Since  the  Audit  Period 

The  following  change  was  made  to  the  Definition  of  Measurement 
in  FY2004: 

Eoch  /ine  will  be  checked  twice  a year.  Such  checks  shall  be  conducted 
no  less  often  than  10  weekdays  and  weekends  per  check.  An  annual 
checking  schedule  shall  be  established  for  the  routes.  The  order  in 
which  the  routes  are  checked  will  be  determined  monthly  throuyjj 
a random  selection  process.  To  the  extent  automated  systems  can 
be  substituted  at  less  cost  for  such  checks,  or  the  measurement 
of  any  performance  standard,  such  systems  must  be  used.  Actual 
headways  against  scheduled  headways  on  all  radial  express,  cross- 
town, sccondory,  and  feeder  tines  for  peak,  base,  evening,  and  late 
night  services.  Will  also  check  lines  during  periods  when 
their  headway  is  greater  than  1 0 minutes. 
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Action  Plan 

Streamlining  the  Service  Standards  and  Improving  the  Definition  of  Measurement 

Headway  adherence  is  typically  used  to  measure  the  reliability  of 
transit  services  operating  at  headways  that  are  10  minutes  or  less.* 
From  a passenger's  perspective,  only  headway  adherence  matters 
when  headways  are  10  minutes  or  less,  since  passengers  do  not  rely 
on  schedules  to  plan  their  trips  because  they  can  rely  on  the  fact 
that  a vehicle  will  arrive  shortiy  after  they  arrive  at  their  stop. 

Given  that  Muni  has  resumed  measuring  headway  adherence  on 
radial  linos  in  FY200d,  it  is  important  to  examine  the  most  effective 
ways  In  whicii  to  measure  headway  adherence  for  high  frequency 
transit  services  (those  with  headway  intervals  of  10  minutes  or 
less).  Headway  adliercnco  is  a less  relevant  reliability  measure  for 
transit  soi  vicos  with  headways  greater  than  10  minutes,  particularly 
if  service  reliability  is  already  being  measured  by  schedule  adherence. 
Consequently,  the  auditor  recommends  that  headway  adherence  be 
measured  only  for  tliosc  transit  services  with  headway  intervals 
of  10  minutes  or  loss,  and  that  scticdule  adherence  (i.e.,  on-time 
performance)  he  measured  for  all  other  transit  services  with 
headway  Intervals  that  are  greater  tlian  10  minutes.  Furthermore, 
we  recommend  that  the  method  of  measuring  headway  adherence 
be  modified  so  that  it  is  more  relevant  to  high  frequency  service. 
Following  are  tlirce  possible  alternatives  for  changes  to  the  method 
of  measurement; 

• AlternAtivc  U\:  The  method  of  headway  adherence  mea- 
surement recommended  in  tlio  Tronsit  Copoc/ty  ond  Quality 
of  Service  Momiol — 2nd  Edition  is  based  on  the  "coefficient  of 
variation  of  headways"  of  transit  vehicles  and  is  calculated  in 
the  following  way: 

Coodicient  ol  vnrinlion  of  honclwnys  ■ 

(stnndmd  deviation  ol  headway  deviallons)+(menn  scheduled  headway)* 


This  calculation  compares  the  difference  between  actual  and 
scheduled  headways  with  that  of  the  scheduled  headway.  A 
lower  coefficient  means  that  there  is  very  little  deviation  in 
actual  headways  from  scheduled  headways  and  that  service 
reliability  is  excellent.  A larger  coefficient  means  that  there  is 
a significant  amount  of  deviation  between  actual  and  scheduled 
headways  and  that  service  reliability  is  poor.  The  table  on 
the  opposite  page  summarizes  information  from  the  Transit 
Capacity  and  Quality  of  Service  Manual — 2nd  Edition  regarding 
coefficient  values  and  associated  levels  of  service  (LOS).  It  is 
important  to  note  that  this  measurement  and  its  associated 
LOS  values  only  applies  to  transit  services  with  headways  of 
1 0 minutes  or  less. 

If  Muni  chooses  to  use  this  measurement  approach,  a goal  of 
LOS  C or  better  should  be  established  for  headway  adher- 
ence. 


LovdI of 
Service 

Coefficient 

Value 

. Comments 

A 

0.00-0.21 

Service  is  provided  like  clockwork,  with  very  regular 
headways. 

B 

0.22-0.30 

Most  vehicles  are  off  the  scheduled  headway  by  a few 
minutes,  but  the  likelihood  of  being  off-headway  by  more 
than  one-half  the  scheduled  headway  amount  is  low  (e.g., 
5 minutes  off  a 10  minute  scheduled  headway). 

C 

0.31  0.39 

Vehicles  are  often  off-headway,  with  a few  headways 
much  longer  or  shorter  than  scheduled. 

D 

0.400.52 

Headways  are  quite  irregular,  with  up  to  one  in  three 
vehicles  one-half  a headway  or  more  off-headway. 

E 

0.53-0.74 

Bunching  occurs  frequently. 

F 

0.75 

Most  vehicles  are  bunched. 

Ffon>  the  fronM  Copoory  ond  Quoky  of  Somco  Monuof — 2nd  EdMn  (2003) 
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• Alternative  #2:  The  approach  used  by  MTA-NewYork  City 
Transit  is  similar  to  that  currently  used  by  Muni,  but  has  a 
specified  time  window  and  focuses  solely  on  incidences  where 
headway  times  are  exceeded.  MTA-NewYork  City  Transit 
measures  the  percentage  of  transit  vehicle  arrivals  where  the 
actual  headway  exceeded  the  scheduled  headway  by  more 
than  3 minutes.  Actual  headways  that  are  less  than  scheduled 
headways  are  not  considered,  based  on  the  assumption  that 
short  headways  generally  result  from  the  previous  vehicle's 
long  headway,  and  that  the  vehicle  with  the  long  headway  is 
the  one  that  affects  passenger  service  quality,  as  a result  of 
longer  wait  times  and  more  crowded  conditions  on  board.* 

• Alternative  #3:  This  alternative  should  only  be  used  if  Muni 
continues  to  measure  schedule  adherence  for  all  lines,  regard- 
less of  their  headway  interval.  While  radial  lines'  frequency 
of  scheduled  service  accommodates  passenger  demand,  the 
reality  of  service  delivery  often  presents  gapping  and  bunching, 
resulting  in  abnormal  headways.  A surrogate  measurement  to 
capture  the  provision  of  capacity  on  major  radial  lines  is  the 
measurement  of  throughput.  This  observation  and  measure- 
ment of  actual  trips  (the  number  of  scheduled  vehicles  that 
pass  a particular  point  within  a given  period  of  time)  allows 
for  a consistent  report  on  trunk  capacity  and  street  condi- 
tions. Both  Service  Standards  la  (on-time  performance)  and 
5a  (peak  period  passenger  load  factors)  measure  components 
of  major  radial  service  delivery,  but  neither  captures  loss  of 
capacity.  MTA  could  consider  substituting  throughput  for 
Service  Standard  6a. The  Proposition  E target  of  80%  should 
be  modified  over  a multi-year  program  to  reach  90%.  The 
correlation  of  this  target  to  that  of  Service  Sundard  2a  ("not 
less  than  98.5  % on  July  I,  2004")  is  imporunt,  so  that  there 
is  an  allowance  for  on-street  loss  of  specific  trips/throughput 
due  to  equipment  malfunaions,  blockages,  etc. 
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Additional  Recommendations  to 
Improve  Performance 

As  was  the  case  with  on-time  performance  in  Service  Standard  la, 
it  is  likely  that  on-street  attention  to  supervision  aiul  preparation  of 
new  schedules  will  be  needed  in  order  to  achieve  headway  adherence 
goals.  For  further  detail,  please  see  the  recommendations  made  witli 
regard  to"Schedulc  Evaluation/Remodiation"  aiuP'Route  Evaluation" 
under  the  Action  Plan  for  Service  Standard  la. 

Somp/ing  Resources  - Reallocation  of  Terminal 
Checking  to  Increased  NTD  Checking 

Although  the  Proposition  E measurement  ret|uircmcnt  included 
checks  at  terminals  and  established  timepoints,  the  checking  resources 
expended  to  collect  from  terminals  Is  about  twice  that  rcc|ulrod  for 
establishcd/intcrmcdlate  timo()Oints.  The  Pro[)ositlon  E mid-point 
sampling  plans  are  coordinated  with  Muni's  NTD  sampling  plans  to 
minimize  the  checking  resources  expended.  The  Data  Collection 
Plan  for  FY2003  (August  1,2002)  Indicates  that  9,720  hours  |>or  year 
of  checking  is  required  for  the  Proposition  E program  at  4.5  locations 
per  line.  Additionally,  3,432  hours  ()cr  year  of  checking  is  required 
for  the  NTD  program  at  two  locations  [;cr  line.  Clo.uiy, .»  significant 
74%  of  the  combined  programs  is  allocated  to  Proposition  E. 

The  resulting  difference  in  schedule  adherence  (between  terminals 
and  intermediate  timepoints)  Is  only  about  I %.  Iiowcver,  Thus,  while 
checking  terminals  provides  a validation  of  the  cliccks  at  intermediate 
timepoints  and  identifies  trips  that  leave  the  terminal  off-scliedulc. 
actual  results  indicate  that  such  a validation  may  not  be  necessary. 
It  would  appear  prudent  to  increase  the  sampling  of  Intermediate 
timepoint  checks  in  lieu  of  the  terminal  checks  with  the  goal  of 
sampling  each  line  each  quarter  rather  than  the  current  twice  yearly 
target 
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The  auditor  recommends  that  the  portion  of  Proposition  E checking 
hours  allocated  to  terminal  checks  (roughly  4,320  hours  per  year) 
be  reallocated  to  mid-point  locations.  The  goal  of  such  a reallocation 
would  be  to  increase  the  frequency  of  checks  to  provide  additional 
seasonal/qiiartcriy  observations  on  each  mode.  Poorly  performing 
rotitos  should  bo  targeted  for  additional  observations  as  well.  If  the 
approximately  4,320  hours  currently  allocated  to  terminal  checks 
were  reallocated  to  the  current  NTD  program  of  3,432  hours,  it 
would  more  than  double  the  frequency  of  checks  made  under  the 
NTD  program. 

Another  op[)ortunlty  for  reallocating  on-time  performance  checking 
resources  could  be  found  by  taking  advantage  of  existing  technology. 
On-time  [)erformance  of  routes  that  are  monitored  by  Next  Bus 
and/or  by  the  ATCS  system  in  the  subway  could  be  done  with 
improved  accuracy  by  taking  advantage  of  those  automated  systems. 
Checker  hours  deployed  on  those  routes  could  bo  used  for  additional 
monicoi'ing  of  targeted  services. 

Coo/'d/f)otion  w/t/]  other  Goals  and  Objectives 

On-time  performance  Is  closely  related  to  the  other  performance 
goals  within  this  grou[i  of  service  standards.  The  steps  taken  to 
improve  on-time  performance  should  be  closely  coordinated  with 
actions  taken  to  achieve  other  performance  measures,  particularly 
those  related  to  schedule  adherence  (la),  delivery  of  scheduled 
service  (2a)  and  safety  (6tl). 

Use  of  Cumulative,  Annual  Performance  vs, 

Qiiorter/y  Performance 

Rather  than  using  fourth-quartoi'  results  to  determine  achievement 
of  annual  goals,  the  atiditor  recommends  that  cumulative  or 
average  annual  performance  he  used  instead.  There  are  a number 
of  internal  and  external  factors  that  load  to  substantial  variation  in 
performance  between  quarters,  including  the  fact  that  diffei'ent  lines 
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are  sampled  each  year  during  the  fourth  quarter.  Reporting  year- 
to-date  or  cumulative  annual  performance  may  provide  a better 
picture  of  systemwide  and  modal  performance  over  the  entire  year 
Quarterly  performance  could  still  be  used  to  determine  variations 
in  progress  toward  meeting  performance  goals  throughout  the  year 
Following  are  both  the  fourth  quarter  and  year-to-date  systemwide 
performance  in  FY200I  and  FY2002. 


FY2001 

FY2002 

4th  Quarter 

YeartoDate 

‘ 4th  Quarter 

Year-to-Date 

Systemwide 

56.8% 

51.8% 

72.1% 

69.0% 

Performance  results  for  this  service  standard  have  been  reported  in 
two  different  formats.  The  4th  Quarter  FY200I  Service  Standards 
Report  contained  a system-wide  table  summarized  by  quarter  which 
included  information  on  headway  adherence  for  each  mode  and  for 
the  entire  system  for  each  quarter.  The  4th  Quarter  FY2002  Service 
Standards  Report  did  not  contain  a systemwide  and  modal  summary 
table.  Because  the  sampling  plan  for  lines/modes  varies  by  month, 
both  a quarterly  and  an  annual  performance  summary  should  be 
presented. 
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Goal: 

1^0  less  than  98.5%  Vehicle  Availabiliiv 

r urposc. 

0 measure  the  percentage  of  equipment  available  for  service 

Definition  of 
Measurement: 

equirement.  An  additional  measurement  is  added  which  shows  the  number  ol 
days  when  a facility  (mode)  does  not  have  100%  ajjaijabilii^ 

ivietnod  of 
Measurement: 

me  veliicle  Maintenance  System  provides  the  data  for  all  modes  with  tho 
exception  of  Cable  Car.  A vehicle  is  considered  available  for  service  if  it  is  available 
jor  assignment  to  an  operator  no  later  than  7:00  a.m.  and  4 00  n m 

ivJiiestones: 

98.5%  (for  both  FY2001  and  FY  2002)  

From  Mum  Service  Siandorxit  Report,  FY200I  and  FT2002  ' 

San  Francisco  Municipal  Raiiway 
Reported  Performance 

For  both  FY200I  and  FY2002,all  modes,  with  the  exception  of  light 
rail,  met  the  goal  of  98.5%  vehicle  availability  for  both  AM  and  PM 
periods.  (For  liglit  rail,  percent  vehicle  availability  was  97.5%  for  AM 
and  97.4%  for  PM  for  the  fourth  quarter  of  FY2002.)The  following 
tables  summarize  the  reported  performance  for  FY200I  - FY2004. 
Veliicle  reliability  (mean  distance  between  failure)  by  mode  Is  not 
rc[)orted  as  [jart  of  this  service  standard,  but  is  instead  reported 
under  standard  9a, "Miles  l)etwccn  roarlcalls  by  mode." 

Although  cumulative  performance  over  tlie  entire  fiscal  year  was 
reported  for  FY200l,only  (luartorly  [)erformancc  was  reported  for 
FY2002.  Average  annual  performance  was  used  to  determine  if  the 
FY200I  goal  was  met,  while  fourth  quarter  performance  was  used 
to  (lotormine  wheihci  or  not  the  FY2002  goal  had  been  met.  (The 
MTA  Board  established  that  |)eiformancc  for  the  entire  fiscal  year 
would  be  measured  by  fourth  (luartcr  performance,  rather  than  by 
performance  over  the  entire  year.)  To  compare  performance  for  the 
two  years,  and  sul)se(|uoiu  years,  only  fourth  quarter  performance 
is  presented  here.  Light  rail  ap[)ears  to  be  consistent  in  its  inability 
to  meet  the  established  goal  during  tlie  fourth  quarter  of  each  fiscal 
year  for  which  data  are  availalde.  Other  modes  and  divisions  have 
met  the  goal  relatively  consistently, 

It  should  l)e  noted  that  in  many  cases  a surplus  of  vehicles  (more  than 
100%)  are  available  for  service.  However,  for  tlie  purposes  of  the 
Service  Standartls  Report,  surplus  availability  was  simply  reported 
as  100%. 

The  definition  of  measurement  states. "An  additional  measurement 
is  added  which  shows  the  number  of  days  when  a facility  (mode) 
does  not  have  100%  availability"This  measurement  was  not  included 
in  either  the  FY200I  or  FY2002  Service  Standards  Reports. 
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Table  A-9  FY200 1 AM/PM  Vehicle  Availability 


. AM&PM 
Vehicle 
, Availability 

IslQ 

AM 

IstQ 

PM 

2ndQ 
AM  ; 

1 - i' 

2ndQ 

PM 

3rd'0  ' 
AM 

■3rdQi 

PM 

,^4th  Q 
AM  ' 

' 4th  Q 
, PM 

' FYOl  ' 

“goal 

Motor  Coach 

99.2% 

99.9% 

94.8% 

96.9% 

98.4% 

99.1% 

99.7% 

99.4% 

1 

98.5% 

riynn-Artic 

99.8% 

99.6% 

94% 

94.3% 

96.3% 

95.7% 

99.7% 

99.2% 

Kirkland 

97.9% 

100% 

90.4% 

96.4% 

96.6% 

100% 

98.9% 

98.4% 

Woods 

100% 

100% 

100% 

100% 

100% 

100% 

100% 

100% 

Trolloy  Coach 

99.8% 

99.4% 

99.7% 

97.9% 

99.9% 

99.5% 

99.8% 

98.8% 

pQlroro  Artie 

99.5% 

98.3% 

99.7% 

94% 

99.3% 

97.4% 

98.9% 

95.0% 

Poiroro 

100% 

100% 

100% 

99.9% 

100% 

99.9% 

99.9% 

97.2% 

Prosidio 

100% 

100% 

99.4% 

99.7% 

100% 

100% 

100% 

100% 

Light  Rail 

98.6% 

98.4% 

96.8% 

96% 

96.2% 

92.5% 

99.9% 

99.5% 

Broda* 

98.8% 

99.4% 

98.2% 

99% 

•• 

•• 

Booing* 

94.8% 

92.8% 

90.7% 

85.9% 

•• 

F lino 

90.7% 

99.2% 

98.4% 

99.1% 

100% 

100% 

100% 

100% 

Cnblo  Cnr 

100% 

100% 

100% 

100% 

100% 

100% 

100% 

100% 

From  Muni  Scnncf  SlondanJi  Rtport.  4th  ^uontr  FY200I 

•Note  Reiulu  for  Breda  and  Boeing  divisions  were  not  reported  separately  after  the  second  quarter. 


Table  A- 1 0 FY2002  AM/PM  Vehicle  Availability 


AM&PM 

Vohicio 

Availability 

IstQ 

AM 

IstQ 

PM 

2ndQ 

AM 

2ndQ 

PM 

3rdQ  ! 
AM 

3rdQ 

PM 

4lhQ 

AM 

4th  Q 
PM 

FY02 

GOAL 

Motor  Coach 

90.3S 

99.IS 

98.3% 

97.7% 

98.1% 

99.9% 

99.9% 

99.8% 

98.5 

Flynn  Ariic 

97.6% 

97.2% 

100% 

100% 

99.9% 

100% 

100% 

100% 

Knkland 

95.3% 

98.7% 

93.2% 

91.2% 

92.6% 

97.4% 

99.7% 

99.5% 

Woods 

100% 

100% 

100% 

100% 

100% 

100% 

100% 

100% 

TrolloyCoach 

99.9S 

98.8% 

99.0% 

97.7% 

9B.5S 

98.9% 

99.2% 

98.6% 

Potrero  Arlic 

99.3% 

98.5% 

98.8% 

96.7% 

98.7% 

98.6% 

100% 

100% 

Potrero 

99.9V 

96.0% 

99.6% 

96.6% 

99.0% 

97.7% 

lOOS 

99.7% 

Prosidio 

100% 

100% 

98.2% 

98.5% 

98.2% 

99.6% 

96.9% 

97.3% 

light  Rail 

100S 

99.2% 

99.3% 

99.0% 

97.3% 

97.1% 

97.5% 

97.4% 

FLins 

100% 

100% 

100% 

100% 

100% 

100% 

100% 

lOOS 

Cable  Car 

lOOS 

100% 

100% 

100% 

100% 

100% 

100S 

100% 

Ffx>m  Aluni  Unrcf  Standards  Report.  4th  Quorttr  FY2002 
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Table  A-l  I FY2003  AM/PM  Vehicle  Availability 


Vehicle 

Availability 

1st 

QAM 

1st 

QPM 

2nd 

QAM 

2nd 

QPM 

3rd  j 
QAM  , 

3rd 

QPM 

4th 

QAM 

4th 

QPM 

FY03 

GOAL 

Motor  Coach 

99.2% 

99.4% 

99.5% 

99.8% 

99.9% 

99.9% 

99.7% 

99.6S 

98.5% 

Ftynn-Anic 

99.9% 

99.8% 

100% 

99.9% 

99.9% 

99.9% 

98.9% 

98.2% 

Kirkland 

97.8% 

99.5% 

100% 

100% 

100% 

99.9% 

99.9% 

98.2% 

Woods 

100% 

99.1% 

99.9% 

99.7% 

100% 

100% 

100% 

100% 

Trolley  Coach 

99.9% 

99.1% 

100% 

100% 

99.9% 

99.8% 

99.9S 

99.8% 

PolreroAnic 

99.5% 

99.3% 

100% 

100% 

100% 

100% 

99.4% 

98.4% 

Poirero 

100% 

97.7% 

100% 

100% 

100% 

100% 

100% 

100% 

Presidio 

100% 

99.9% 

100% 

100% 

99.9% 

99.6% 

100% 

100% 

Light  Rail 

96.8% 

96.5% 

99.3% 

98.9% 

99.2% 

98.5% 

98.6% 

97.8% 

1 F Line 

100% 

100% 

98.1% 

99.3% 

99.8% 

99.7% 

100% 

100% 

1 Cable  Car 

100% 

100% 

100% 

100% 

100% 

100% 

100% 

100% 

m Mum  Service  Standanii  Report.  Quarter  FY  2003 
au  in  iUlia  have  not  been  audited. 


able  A- 1 2 FY2004  1st  Quarter  AM/PM  Vehicle  Availability 


AM  & PM  Vehicle  , 
Availability  ; 

1st  QAM 

1st  QPM 

FY04GOAL 

Motor  Coach 

99.8% 

99.3% 

98.5% 

Flynn  Artie 

99.4% 

97.2% 

Kirkland 

100% 

100% 

Woods 

100% 

100% 

Trolley  Coach 

99.9% 

99.9% 

PotreroArtic 

99.4% 

99.3% 

Potrero 

100% 

100% 

PresHbo 

100% 

100% 

Light  Rail 

92.3% 

92.3% 

FLino 

98.8% 

99.8% 

C«ble  Car 

100S 

100% 

Frorn  Semcc  Uandardt  Report.  I si  Quongf  PY2004 
Data  In  robes  not  bowi  audited 
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Venficot/on  of  Definition  and 

Method  of  Measurement  During  the  Audit  Period 

For  both  FY200I  and  FY2002,  data  were  collected  and  aggregated 
according  to  the  definition  and  method  of  measurement  included 
in  the  Service  Standards  Reports.  Vehicles  are  unavailable  during 
repairs  or  maintenance  and  arc  considered  to  bo  "on  hold"  if  they 
are  unavailable  for  service  for  longer  than  a one-hour  period  of  time. 
When  repairs  or  maintenance  arc  completed,  the  vehicle  is  taken 
off  hold.  All  vehicle  maintenance  data  arc  ctucrcd  into  the  Vehicle 
Maintenance  System  (VMS).  Operations  personnel  are  instructed 
to  save  the  hold  count  in  VMS  each  morning  and  afternoon.  VMS 
automatically  calculates  the  daily  percentage  availability  using  this 
information. 

Daily  percentage  availability  data  was  then  downloaded  from  VMS 
and  entered  into  spreadsheet  software  In  order  to  calculate  monthly 
and  quarterly  availability  for  each  division  and  mode.  To  calculate 
the  overall  total  vehicle  availability  for  the  motor  coach  and  trolley 
modes,  a weighted  average  based  on  the  number  of  vehicles  that 
were  within  each  motor  coach  and  trolley  division  was  used. 

Vcnficat/on  of  Reported  Performance 
During  the  Audit  Period 

Based  on  an  audit  of  the  monthly  and  quarterly  VMS  reports  for  tlic 
AM  percent  vehicle  availability  data  for  FY2002,  data  appear  to  bo 
reported  correctly. 
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San  Francisco  Municipal  Railway 

Problems  or  Issues  Identified  During  the  Audit  Period 

No  issues  or  problems  were  found  with  the  method  of  measurement, 
however  several  problems  with  both  reported  performance  and 
tlie  definition  of  measurement  were  found.  Reported  performance 
was  not  consistent  between  quarterly  Service  Standards  Reports 
in  FY200I.  For  FY200l,first  quarter  performance  reported  in  the 
1st  Quarter  FY2001  Service  Standards  Report  did  not  match  first 
c|uarter  performance  reported  in  the  2nd  Quarter  FY200I  Service 
Staiulards  Rc|)ort.  Similarly,  third  quarter  performance  reported  in 
the  3rd  Quarter  FY200I  Service  Standards  Report  did  not  match 
reported  third  (|uartcr  performance  in  the  4th  Quarter  FY2001 
Service  Standards  Ro[)ort.  For  FY2002,  performance  was  reported 
consistently  between  (|uartcrs. 

For  both  FY200 1 and  FY2002.  the  definition  of  measurement  was  not 
followed  completely.  Part  of  the  definition  of  measurement  states, 
"An  additional  measurement  is  added  which  shows  tlic  number  of 
days  when  a facility  (mode)  does  not  have  100%  availability."  This 
information  was  not  reported  in  either  the  FY200I  or  the  FY2002 
quarterly  Service  StandarcIs  Reports.  While  tills  statement  was 
removed  from  the  FY2003  Service  Standards  Reports.it  was  included 
but  not  reported  in  the  FY2004  First  Quarter  report. 
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Changes  in  Measurement  Since  the  Audit  Period 

Subsequent  to  the  audit  period,  Muni  has  made  no  changes  to 
the  definition  of  measurement,  method  of  measurement,  nor  the 
milestones. 

Auditor  Recommendations 

Rather  than  using  fourth  quarter  results  to  determine  achievement 
of  annual  goals,  the  auditor  recommends  that  cumulative  or  average 
annual  performance  be  used  instead.  There  are  a number  of  internal 
and  external  factors  that  lead  to  variation  in  performance  between 
quarters.  Reporting  year-to-date  or  cumulative  annual  performance 
may  provide  a better  picture  of  systemwide  and  modal  performance 
over  the  entire  year.  Quarterly  performance  could  still  be  used  to 
determine  variations  in  progress  toward  meeting  performance  goals 
throughout  the  year. 

It  is  difficult  to  ascertain  from  this  measure  whether  failures  to  meet 
98.5%  or  100%  vehicle  availability  are  due  to  an  insufficient  number 
of  spare  vehicles,  or  an  overwhelming  number  of  mechanical  failures, 
or  a combination  of  both.  Reporting  information  on  spare  ratios  (the 
number  of  spare  vehicles  divided  by  the  fleet  size)  for  each  mode 
and  division  could  provide  useful  information  about  the  efficiency  of 
fleet  utilization  and  the  underlying  causes  of  failures  to  meet  goals 
for  percent  vehicle  availability. 

Finally,  the  definition  of  measurement  should  be  amended  to  remove 
the  statement,  "An  additional  measurement  is  added  which  shows 
the  number  of  days  when  a facility  (mode)  does  not  have  100% 
availability."  or  this  information  should  be  included  in  the  Service 
Standard  Reports. 
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8a.  Unscheduled  absences  by  operator,  mechanical  and 
administrative  personnel. 


Goal: 


Annual  5%  reduction  of  YTO  for  Maintenance  (Mechanical),  10%  reduction  lor  Transportation 
(Operators),  and  5%  reduction  for  Administration. 


Purpose: 


To  measure  unscheduled  absences. 


Definition  of 
Measurement: 


Monthly  measurement  of  unscheduled  absences  is  a defined  as  time  that  is  not  scheduled  in 
advance  and  includes  the  following  payroll  categories: 

Method  of  Measurement':  Sick  pay  (with  pay),  Sick  Leave  (without  pay),  AWOL,  Worker's  Comp. 
SDI  (long  term  illness),  and  Assault  Pay.  The  online  Attendance  Tracking  System  currently 
provides  the  data  as  a calculation  of  scheduled  hours  available  against  unschcdulod  hours. 
Unscheduled  absences  is  tracked  for  operators,  mechanical  and  administrntivo  staff  by  mode. 


Method  of 
Measurement: 


Milestones: 


FY2001 


FY2002 


Division 

Goal 

Division 

Goal 

Maintenance 

5%  Reduction  of  FYTD 

Maintenance 

5%  Reduction  of  FYTD 

Operations 

10%  Reduction  of 
FYTD 

Administration 

5%  Reduction  of  FYTD 

Administration 

5%  Reduction  of  FYTD 

Operations 

5%  Reduction  of  FYTD 

Transit  Operators 

10%  Reduction  of  FYTD 

From  Atofw  Service  UandonH  Aeport,  F>700/  and  FY2002 
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San  Francisco  Municipal  Railway 


Reported  Performance 

Unscheduled  absence  hours  were  reported  as  a percentage  of  the 
total  sclieduled  hours  for  each  division.  For  FY200I,  operations 
absences  included  both  transit  operators  as  well  as  other  operations 
support  staff.  In  FY2002,  these  two  divisions  were  reported  separately, 
and  different  milestones  were  established  for  each. 

As  shown  In  the  following  table,  all  three  divisions  failed  to  meet  their 
percentage  goals  for  FY200I.  While  the  Administration  Division 
failed  to  meet  its  established  goal  again  in  FY2002,all  other  divisions 
either  mot  or  did  better  than  their  established  goals  in  FY2002. 
All  divisions  wore  able  to  meet  their  goals  in  FY2003.  Table  A- 1 3 
summarizes  the  percentage  of  unscheduled  absences  for  FY200I 
- FY2003. 


T.iblo  A- 1 3 Porcontago  of  Unscheduled  Absences, 
FY200I-FY2003 


Tfom  Muni  Spivkr  Qumlef  fViOO/,  fyiOO?,  FYJ003  ond  /it  Qtwftcf  FY2004 

Oairt  In  Itnilcx  h.wo  not  bpon  avkIUpiI. 

■^NcupiTi  .tnipoi  uilon  InrUidmi  both  Opei  Atlom  Ami  Ti  ai^iII  Op©i  Atori  in  TYIOOI 


Ver/ficat/orj  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

Sciioduloil  ami  unsclKHlulod  liours  arc  tracked  in  two  automated 
timekoepinj’  systems  — ATS  and  TESS.  TESS  is  a relatively  new 
timekeeping  system,  and  hours  for  maintenance,  administrative  and 
operations  personnel  is  tracked  in  this  system.  However,  some 
hours  for  maintenance  and  operations  are  still  tracked  in  ATS.  and  all 
transit  operatoi'  hours  arc  tracked  in  ATS. 


Verification  of  Reported  Performance  During  the 
Audit  Period 

Based  on  an  audit  of  data  for  the  third  quarter  of  FY2002,  all  data 
appear  to  be  reported  correctly. 

Problems  or  Issues  Identified  During  the  Audit  Period 

There  were  no  problems  or  issues  with  regard  to  reported 
performance,  or  the  definition  or  method  of  measurement. 
Tabulating  unscheduled  and  scheduled  hours  data  from  two  different 
timekeeping  systems  is  time-consuming,  however,  and  would  be 
simplified  if  all  timekeeping  was  done  within  one  database. 

Changes  in  Measurement  Since  the  Audit  Period 
The  Method  of  Measurement  was  changed  in  FY2003  to; 

The  online  Attendance  Tracking  System  provides  the  data  as  a 
calculation  of  scheduled  hours  available  against  unscheduled  hours 
for  Transit  Operators.  The  TESS  Timekeeping  System  provides 
the  unscheduled  absences  for  Maintenance,  Administrative  and 
Operations  employees.  Unscheduled  absences  are  tracked  for 
operators,  mechanical  and  administrative  staff  by  mode. 

Auditor  Recommendations 

In  terms  of  service  reliability,  only  a significant  number  of  unscheduled 
absences  of  Maintenance  and  Operations  staff  will  negatively  impact 
service  provision.  Additionally,  the  percentage  of  unscheduled 
absences  for  the  Administration  Division  is  the  lowest  for  all  divisions 
measured.  Consequently,  the  goal  for  reduction  of  unscheduled 
absences  for  the  Administration  Division  should  be  relaxed,  and  a 
more  reasonable  target  should  be  established. 
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9a.  Miles  between  roadcalls  by  mode. 


Goal: 

Increase  the  miles  between  roadcalls. 

Purpose: 

Measure  reliability  through  the  miles  a vehicle  travels  between  failures. 

Definition  of 
Measurement: 

Monthly  measurement  is  currently  dictated  by  the  Federal  Transit  Administration  ns  tallows: 
"Failures  are  classified  as  either  a major  or  minor  failuio  of  an  elLMiient  of  the  vehicle's  mechanical 
system.  For  each  incident  of  a major  or  minor  failure,  report  whether  the  vehicle  completes  tim 
trip  or  the  vehicle  docs  not  complete  the  trip.  If  the  failure  occurs  during  dendhoad  or  layover, 
include  this  in  revenue  vehicle  system  failures.” 

Method  of 
Measurement: 

Data  is  collected  and  input  into  the  online  Vehicle  Maintenance  System  for  all  rovoiuio  vehicles 
except  for  Cable  Car.  which  has  its  own  internal  tracking  system.  Reports  are  gonornted  and 
the  data  for  roadcalls  is  analyzed  against  the  Central  Control  log.  All  verifiable  major  and  minor 
mechanical  defects  are  included  as  part  of  the  Moan  Distance  between  defects  number.  Arons 
that  do  not  result  in  a chargeable  roadcall  to  the  maintenance  shops  include  accidents,  sick 
passengers,  vandalism,  body  damage  and  broken  windows. 

Milestones: 

Milestones:  Increase  miles  between  failures  (sec  table). 

From  Mum  Semce  Standards  Report,  FY200i  and  FY2002 
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Reported  Performance 

Although  cumulative  performance  over  the  entire  fiscal  year  was 
reported  for  FY200I.  only  quarterly  performance  was  reported  for 
FY2002,  and  fourth  quarter  performance  was  used  to  determine 
whether  the  FY2002goal  had  been  met.  (The  MTA  Board  established 
that  performance  for  the  entire  fiscal  year  would  be  measured  by 
fourth  quarter  performance,  rather  than  cumulatively.) 

In  FY200I,  the  Kirldand  motor  coach,  Standard  Potrero,  Presidio, 
Boeing  Light  Rail,  Breda  Light  Rail  and  Cable  Car  divisions  all  met 
their  fiscal  year  goals  at  tlie  end  of  the  fourth  quarter  In  FY2002,  the 
Flynn-Articulated,  Presidio,  Boeing  Light  Rail,  Breda  Light  Rail,  and 


Table  A- 1 4 Milos  between  Roadcalls  (Mean  Distance  between  Failures),  FY200 1 - FY2004 


1 1 

FY2001  1 

1 

[ 

FY2002 

FY2003 

FY2004 

Mode  and  DIvlilon 

4th  Quarter 

FYTD 

Goal 

4th  Quarter 

Goal 

4th  Quarter 

FYTO 

Goal 

1st  Quarter 

Goal  1 

1 Motor-Cnnch  - Monn  Distnncu  Bnlwncn  Fnilurcs  (Mites)  | 

rivnn-Artic 

637 

873 

1,500 

1,929 

1500 

3,033 

2,299 

1,750 

2,279 

1 2,000 

Woods 

1,773 

1.002 

3,000 

1,760 

3M0 

2,802 

2,176 

2,500 

3,014 

2,750 

Kiikliiml 

3,467 

3,138 

3.000 

2,381 

3,000 

1 3,992 

2.918 

3.000 

4,455 

3,000 

Ttullnv  Conch  - Monn  Distanco  Botwonn  Fnilurcs  (Miles) 

• 

■ • . ■- 

Artie  Pntrmo 

443 

509 

500 

508 

750 

583 

541 

650 

630 

675 

Stnmlmd  Potrnio 

601 

737 

550  1 

795 

800 

861 

762 

1,000 

860 

1,000 

Pinsidio 

~'l,375 

1,002 

550 

1,223 

800 

1,221 

1,279 

1,000 

1,280 

1,250 

1 Rnil  - Mean  Distnnen  Bntwnon  Fnilurcs  (Milos) 

• 

• 

Bocinii  Liiiht  Rnil 

1,706 

1,447 

1,300 

Fleet  Retired  [ 

Brmln  Li[]ht  Ihiil 

3,271 

2,246 

1.800 

3,276 

1,900 

3,128 

3,328 

3,000 

3,428 

3,500 

PCC 

608 

905 

1,250 

1,496 

1,250 

1,148 

1,309 

1,250 

1,034 

1,250 

1 Cnble  Cm  - Mean  Distnneo  Between  Fnilurcs  (Mi 

les)  1 

1 1 5.620 

4.809 

2,530 

5,631 

3.000 

6,387 

5658 

5,000 

6,611 

5,500 

Pix>m  Mum  Senkt  n200l,  niOOl  fY^OOi  oth/  /jt  Ooortw  F)f2004 
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Cable  Car  divisions  all  met  their  fiscal  year  goals  at  the  end  of  the 
fourth  quarter.  In  FY2003,  most  divisions  were  able  to  meet  their 
goals  during  the  fourth  quarter,  however  fewer  were  able  to  meet 
their  goals  with  regard  to  cumulative  annual  performance.  Table  A- 
14  summarizes  this  information. 
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Proposition  E Municipal  Transportation  Quality  Review 

Verification  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

Mean  distance  between  failures  (MDBF).or  miles  between  roadcalls. 
data  were  collected  and  tabulated  according  to  the  established 
definition  and  method  of  measurement,  however  roadcall  data  were 
tabulated  directly  from  the  Central  Control  log  and  mileage  data 
were  tabulated  from  the  Vehicle  Maintenance  System  (VMS)  for  all 
modes  except  cable  car.  Following  is  a more  detailed  description  of 
data  collection  and  aggregation  procedures  used  for  each  division. 
All  divisions  used  the  number  of  FTA-reportable  roadcalls  (major  or 
minor  mechanical  failures)  to  determine  MDBF. 

Quarterly  MDBF  represents  the  average  MDBF  data  for  each  month 
within  the  quarter,  except  for  LRV  Most  LRV  quarterly  calculations 
were  made  by  dividing  the  total  mileage  for  the  quarter  by  the  total 
number  of  roadcalls.  rather  than  averaging  the  MDBF  for  each  month 
within  the  quarter. 

Motor  Coach 

Prior  to  October  2001,  roadcalls  were  logged  directly  with  the 
Motorcoach  division.  During  this  time,  the  number  of  reported 
roadcalls  entered  into  VMS  was  not  checked  against  the  Central 
Control  log.  and  it  was  likely  that  the  number  of  roadcalls  entered 
into  VMS  was  slightly  less  than  the  actual  number  of  roadcalls  as  a 
result.  After  October  2001,  roadcall  data  were  downloaded  from 
the  Central  Control  log  on  a daily  basis.  The  Central  Control  log 
was  compared  to  the  repair  log  to  determine  the  actual  number 
of  failures.  (Occasionally,  reported  failures  were  determined  to  be 
inaccurate  after  maintenance  staff  found  no  mechanical  defects.) 
Spreadsheet  software  was  used  to  tabulate  the  number  of  roadcalls 
and  MDBF  for  each  quarter  during  Pf 200 1 and  FY2002. 
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Vehicle  mileage  was  also  tracked  using  VMS.  All  vehicles  used  hub 
odometers,  and  mileage  counts  were  electronically  uploaded  directly 
into  VMS.  Mileage  data  was  double  checked  against  fuel  usage  data 
to  determine  any  inaccuracies  or  mistakes. 


Poirero  Trolley 

Roadcall  data  were  downloaded  from  the  Central  Control  log  and 
tabulated  either  by  hand  or  using  spreadsheet  software.  Mileage 
data  obtained  from  eitlier  dash  or  hub  odometers  were  entered 
into  VMS  and  double  checked  by  maintenance  staff.  Spreadsheet 
software  was  used  to  calculate  MDBF  using  roadcall  data  from  the 
Central  Control  log  and  mileage  data  from  VMS. 

Presidio  Trolley 

Prior  to  March  2001,  roadcall  data  was  obtained  dally  from  the 
Central  Control  log  and  was  totaled  for  each  month  and  quarter. 
After  March  2001,  tlio  division  used  its  own  dally  roadcall  log  to 
determine  the  number  of  roacicalls  or  failures, 

Maintenance  staff  recorded  mileage  dally  on  a handwritten  log  and 
then  entered  It  into  VMS.  A supervisor  double-chocked  all  mileage 
entries.  MDBF  was  calculated  monthly  by  hand  using  totals  from 
printouts  of  the  Central  Control  log  (for  roadcalls)  and  VMS  (for 
mileage). 
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Light  Rail  Vehicles 

Roadcalls  were  recorded  by  maintenance  personnel.  Only  roadcalls 
that  required  more  than  five  minutes  to  address  (from  arrival  to 
resolution)  and  were  FTA-reportabIc  were  counted  toward  the 
MDBF  calculation.  In  FY200I  and  FY2002.  LRV  roadcalls  did  not  go 
through  Central  Control,  but  were  tracked  by  the  division.  The  PCC 
maintenance  ci'cw  also  tracked  their  roadcalls.  Roadcall  data  from 
PCC  and  LRV  maintenance  was  cross-checked  for  any  duplications, 
and  then  tabulated. 

For  light  rail  vehicles  other  than  historic  PCC  cars,  mileage 
was  tracked  daily  and  then  entered  into  VMS.  For  historic  PCC 
cars  (which  do  not  have  odometers),  staff  calculated  mileage  by 
multiplying  the  number  of  service  hours  by  9.5  miles  per  hour 
(which  is  assumed  to  be  the  average  travel  speed  for  these  vehicles). 
Staff  used  sfireadshcet  software  to  calculate  MDBF  from  reported 
roadcall  and  mileage  data. 

Cable  Car 

Because  cable  cars  do  not  have  odometers,  staff  calculated  the  actual 
fleet  mileage  by  sul)tracting  tiic  number  of  miles  lost  due  to  missed 
trips  and  chargcal)lo  roadcalls  from  tlie  scheduled  mileage  (scheduled 
trips  per  day  multiplied  by  the  miles  per  trip)  for  each  day. 

Roadcalls  fi  om  tlio  roadcall  log  were  recoi'dcd  into  tlic  daily  morning 
report.  Daily  reports  were  used  to  calculate  the  number  of  roadcalls 
per  montli.  MDBF  calculations  for  each  month  were  made  by  hand 
using  the  calculated  mileage  anti  roadcall  data. 
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Verification  of  Reported  Performance  During  the 
Audit  Period 

Motor  Coach 

Based  on  an  audit  of  FY200I  data,  only  the  reported  performance 
data  from  the  third  quarter  matched  the  original  data  provided  by 
the  division.  For  the  first,  second  and  fourth  quarters  of  FY2001, 
source  data  did  not  match  what  was  reported  in  the  4th  Quarter 
FY200I  Service  Standards  Report.  The  following  discrepancies  were 
found: 


Mean  Distance  Between  Failures  (Miles)  Reported  and  Audited  Data 


Division 

Reported  i 

' Audited  ' 

Reported 

1 Audited 

' Reported 

Audited 

FlynnArtic 

931 

870 

916 

920 

837 

942 

Woods 

1,840 

1,847 

2,041 

2,074 

1,773 

1,826 

Kirkland 

3,119 

2,925 

3,090 

3,113 

3,467 

3,438 

Potrero  Trolley 

Based  on  an  audit  of  data  for  the  first  and  fourth  quarters  of  FY2002 
for  both  the  Standard  and  Artie  Divisions,  data  appear  to  be  reported 
correctly. 

Presidio  Trolley 

Based  on  an  audit  of  data  for  the  first  quarter  of  FY200 1 . data  appear 
to  be  reported  correctly. 


I Proposition  E Municipal  Transportation  Quality  Review 
Light  Rail  Vehicles 

For  FY200 1 , quarterly  MDBF  data  for  each  of  the  four  quarters  were 
calculated  by  dividing  the  total  mileage  for  the  quarter  by  the  total 
number  of  roadcalls  that  occurred  within  the  quarter.  For  FY2002. 
this  same  calculation  method  was  used  to  determine  the  MDBF  for 
the  first  and  fourth  quarters,  however  the  MDBF  for  the  second  and 
third  quarters  was  calculated  by  averaging  the  MDBF  for  the  months 
within  each  quarter. 

The  FY2002  third  quarter  MDBF  reported  for  the  entire  Light  Rail 
Division  does  not  match  source  data.  According  to  the  3rd  Quarter 
FY2002  Service  Standards  Report,  the  FY2002  third  quarter  MDBF 
I for  the  Light  Rail  Division  was  2.237.  According  to  source  data,  the 
average  MDBF  for  the  Light  Rail  Division  should  be  2.657  for  the 
' third  quarter  of  FY2002. 

I Cable  Car 

Based  on  an  audit  of  data  for  the  fourth  quarter  of  FY2002.  data 
•I  appear  to  be  reported  correctly. 

Problems  or  Issues  Identified  During  the  Audit  Period 

Quarterly  MDBF  data  were  not  calculated  in  a consistent  manner 
for  light  rail  vehicles.  In  FY2002.  first  and  fourth  quarter  data  were 
calculated  by  dividing  the  total  mileage  for  the  quarter  by  the  total 
number  of  roadcalls  that  occurred  within  the  quarter.  The  MDBF 
for  the  second  and  third  quarters  was  calculated  by  averaging  the 
MDBF  for  the  months  within  each  quarter.  In  addition,  numerous 
calculation  errors  were  identified.  It  is  possible  that  audited  source 
data  was  adjusted  in  some  way  prior  to  reporting;  however,  no 
documentation  of  chose  adjustments  were  identified. 
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Changes  in  Measurement  Since  the  Audit  Period 

Subsequent  to  the  audit  period.  Muni  has  made  no  changes  to 
the  definition  of  measurement,  method  of  measurement,  nor  the 
milestones. 

Auditor  Recommendations 

Quarterly  MDBF  should  be  calculated  in  a consistent  manner  - 
either  by  dividing  the  total  mileage  by  the  total  number  of  roadcalls, 
or  by  averaging  the  MDBF  for  the  three  months  within  a quarter. 
A description  of  the  calculation  method  should  be  added  to  the 
section  describing  the  method  of  measurement. 

Rather  than  using  fourth  quarter  results  to  determine  achievement 
of  annual  goals,  the  auditor  recommends  that  cumulative  or  average 
annual  performance  be  used  instead.  There  arc  a number  of  Internal 
and  external  factors  that  lead  to  variation  in  [jcrformancc  between 
quarters.  Reporting  year-to-date  or  cumulative  annual  ptM  formance 
provides  a better  picture  of  systemwide  and  modal  performance 
over  the  entire  year.  Quarterly  performance  could  still  be  used  to 
determine  variations  in  progress  toward  meeting  performance  goals 
throughout  the  year. 
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Proposition  E Municipai  Transportation  Q^ty  Review 

B.  System  Performance 

Transit  system  performance  measures  assess  how  well  transit  agency 
resources  are  being  utilized,  how  efficiently  service  is  provided 
within  constraints,  how  effectively  transportation  demand  is  met, and 
how  well  the  agency  is  administered.'  Proposition  E included  the 
following  service  standards  to  tncasure  Muni's  system  performance: 

• II).  Passengers  carried  by  mode. 

■ 2b.  Faro  Revenue  generated  by  mode. 

• 31).  I lours  and  miles  operated  by  mode. 

• 41).  Expenses  incurred  by  mode. 

Key  Recommendations 

The  following  arc  key  recommendations  based  on  the  findings  of  our 
audit  of  FY200I  and  FY2002  Service  Standards  data  and  on  trends 
observed  in  unaudited  FY2003  and  FY2004  Service  Standards  data: 

• For  Service  Standard  I b,  realistic  goals  for  ridership  should  be 
established  l)ascd  on  economic  conditions  as  well  as  available 
service. 

• Fare  revenue  is  an  important  management  too!  that  can  serve 
as  a surrogate  for  ridership  during  periods  of  fare  stability. 
Combined  with  otlior  indicators,  it  is  possible  to  calculate 
average  faro  pci'  passenger,  a useful  management  tool  when 
looking  at  alternative  fare  structures.  Wile  this  indicator 
should  he  continuctl,  the  cui'rent  practice  of  clianging  the 
benchmark  for  faro  rovonuo  annually  based  on  tlie  prior  years 
actual  experience  does  not  allow  for  reasonable  comparison 


'"AGuklrbook  (oi  DosrlvYloj;  aT«  Amit  Prt^oniuiKrvMcvAUii'onioni  Syiicov"  TCRP Report  88, 2003. 
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across  time.  Rather  than  continuing  to  alter  the  benchmark, 
it  is  recommended  that  the  milestone  be  restated: “Fare  rev- 
enue should  increase  by  1.5%  per  year  compared  with  prior 
year  performance.”  Changing  the  milestone  in  this  way  will 
allow  for  comparisons  over  time,  and  will  serve  as  a better 
indicator  of  the  need  to  adjust  fares  and  fare  structure. 

Miles  and  hours  of  service  are  a direct  reflection  on  the  avail- 
able budget.  It  is  not  reasonable  to  expect  that  in  times  of 
budget  constraints  miles  and  hours  of  service  will  continue  to 
increase,  or  even  remain  constant.  Miles  and  hours  of  service 
are  important  inputs  to  the  NationalTransit  Database  (NTD) 
reporting,  and  will  be  calculated  annually  whether  or  not  they 
continue  to  be  used  as  a performance  measure.  However, 
it  is  recommended  that  this  NOT  be  continued  as  a service 
standard  at  this  time. 

The  goal  and  information  presented  for  Service  Standard  4b 
has  limited  value  to  the  public  or  to  Muni  itself.  In  fact,  Muni 
has  no  opportunity  to  exceed  its  budget  without  receiving 
a supplemental  appropriation,  as  a City  agency  cannot  oper- 
ate in  deficit.  Rather  than  comparing  expenses  to  budget  by 
mode,  which  is  an  artificial  comparison,  it  is  recommended 
that  Muni  crack  fully  allocated  costs  per  hour  of  service  by 
mode.  This  standard  industry  benchmark  would  allow  Muni 
to  compare  its  unit  costs  against  other  similar  properties 
nationwide.  This  information  is  calculated  by  the  Finance 
Department  regularly  for  the  NTD  report,  and  would  not 
require  additional  effort. 
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lb.  Passengers  carried  by  mode. 


Goal: 

To  increase  the  number  of  passenger  boardings  by  mode  by  2%. 

Purpose: 

Measurement  of  the  ridership. 

Definition  of  Measurement: 

Annual  measurement  of  the  number  of  passengers  who  board  the  Muni's  revenue  vehicles.  A passenger 
is  counted  each  time  they  board  a vehicle,  even  thougli  they  may  ho  on  the  same  journey  from  oiigin  to 
destination.  Data  will  be  reported  to  the  board  on  a quarterly  basis. 

Method  of  Measurement: 

Ride  checkers  are  utilized  to  count  the  passenger  boardings. 

1 Milestones: 

2%  increase  over  previous  year 

From  Muni  ServKe  Stondords  Report.  fY  2001  ond  FY2002 


Reported  Performance 

Ridership  data  are  reported  annually,  with  data  from  the  previous 
year  reported  for  the  current  fiscal  year  (i.e.,  for  the  FY2002  Service 
Standards  Report,  ridership  for  FY200I  was  reported).  This  is  due 
to  the  fact  that  final  ridership  calculations  cannot  be  completed  until 
several  months  after  the  conclusion  of  the  fiscal  year. 

For  both  FY2000  and  FY200I.  systemwide  ridership  increases 
exceeded  the  goal  of  2%  growth  from  the  previous  year.  However, 
this  growth  was  not  distributed  evenly  among  all  modes.  The  most 
significant  ridership  gains  were  for  light  rail  vehicles,  while  cable 


cars  lost  ridership  during  both  years.  Motor  coach  i itlci  sliip  also 
fell  between  FY2000  and  FY2001.  Between  FYI999  and  FY2001. 
ridership  on  all  modes,  except  for  cable  car.  increased  by  more  than 
2%.  Ridership  decreased  by  7%  between  FY200I  and  FY2002.  and 
decreased  by  1.3%  between  FY2002  and  FY2003.  (This  decrease  is 
most  likely  due  to  the  overall  economic  downturn  that  occuiicd 
during  this  period  of  time.)  Table  B-l  summarizes  ridoiship  anfl 
percent  change  in  ridership  for  fiscal  years  1999,  2000,  2001, 2002 
and  2003. 


Table  b-l  n 

uni  Ktaersn 
FY1999  ' 

ip,  r T 1 TT7 

FY2000 

Percent 

Change 

(FY99FY00I 

FY2001 

Ferceiit 

Change 

(FYOOFYOll 

Percent 

Ctiangu 

IFY99fY01l 

FY  2002 

IY  2003 

Percent 
Chnnyu 
(FYIJ1021  . 

Percent  > 
Change 
(FY02-031 

92  978  418 

96  394,514 

3.7% 

96.032.546 

■0.4% 

3.3% 

92.269,201 

90,8B0,D79 

■3.9%" 

■1.5% 

78  275  162 

78  460,995 

0.2% 

80,868.518^ 

3.1% 

3.3^ 

73.968.384 

74.398, 946 

8.5% 

0.6% 

35  659  856 

41,610,040 

16.7% 

49,698.816 

19.4% 

39.4% 

44.976.4/4 

42.896,209 

•0,5% 

•4.6% 

9 701  321 

9 206,298 

•5.1% 

8.312.945 

■9.7% 

14,3% 

7.257.683 

7.4 18. 790 

•12.7% 

2,2% 

TOTAL 

216.614.757 

225.671.847 

4.2% 

234.912.825 

4.1% 

8.4% 

218.461,742 

216.694.683 

•7.0% 

• 1.3% 

From  Mun,  S*f«*  SwMt  Report.  4ib  Qu»tef  fY200l.  FTlOOl.  m003  and  lu  Quorur  m004 
Diu  m mi*cs  hev«  noc  boon  njdittd 
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Verification  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

Annual  ricicrship  was  estimated  using  Muni's  standard  ride  check 
tT  o LoLc  .-1.10  cl.ed.s  scheduled  » occur  ever,  four 
Baseline  ride  checks  were  ac.uall,  conducted  about  o.er,  ll.e 
Lrs  d.ie  to  stall  tur..o.er.  (In  other  words,  the  basel.no  eldership 
(or  a particular  line  would  be  reesbal.lished  with  cnpincal  data  on  y 
e,ery  live  years.)  Lines  that  h.ad  not  been  checked  with, n the  I s 
four  yea,  s we,  e ji.en  p,-lority  to  be  re-checked.  as  were  lines  that 
underwent  some  otlier  type  of  significant  change  (eithei  a route 
cliange,  or  significant  cliange  in  land  use  along  the  route).  Nearly  all 
linos  ii.ivo  liiul  hasolinc  chocks  since  1998. 

Ricio  checkers  carry  a handhold  electronic  device  which  they  use 
to  count  hoardings  and  alightings.  Typically,  one  checker  is  ■’'ssigned 
to  a vehicle.  Counts  are  electronically  uploaded  from  the  handheld 
devices  onto  computers.  Specialized  software  is  then  used  to 
calculate  loads  and  clieck  tltat  hoardings  and  alightings  balance  out. 
Checkers  also  count  tlie  numhor  of  trips  that  occur  within  a given 
time  period.  This  Information  is  entered  into  the  computer  software 
as  well.  The  daily  hoardings  during  a given  time  period  are  calculated 
hy  multiplying  the  average  hoardings  per  trip  and  tlic  actual  number 
of  trips  that  occurreil  within  a given  time  period. 

R.iw  data  are  edited  before  baselines  are  calculated.  The  RUCUS 
software  j^rogram  (developeil  specifically  for  Muni)  is  used  to 
run  the  baseline  calculations.  B.tscline  data  are  used  to  develop 
a conversion  factor  allowing  ridership  at  the  maximunt  load  point 
on  a line  to  he  converted  to  a ridership  estimate  for  an  entiie 
line.  This  methoilology  has  been  approved  by  the  Federal  Transit 
Administration  (FTA).  During  the  periods  between  baseline  checks, 
checks  are  only  maile  at  maximum  load  points. 
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During  the  reporting  period,  each  line  was  checked  at  its  maximum 
load  point  (MLP)  for  two  days  from  5 AM  - 7 PM.  Some  lines  were 
checked  only  at  one  MLP,  while  others  were  checked  at  several  MLPs 
(different  points  were  checked  for  different  time  periods  for  some 
lines).  Because  the  software  system  only  recognizes  published  time 
points  the  MLP  was  chosen  to  correspond  to  a time  point  as  close 
as  possible  to  the  true  MLP.  The  two-day  check  data  was  averaged, 
and  the  average  load  and  number  of  trips  were  then  calculated. 

The  computer  software  automatically  calculated  the  number  of  trips 
that  needed  to  be  checked  for  statistical  significance.  Extra  checks 
were  often  made  in  order  to  ensure  a valid  count. 

Annual  ridership  calculations  for  individual  lines  are  subject  to  a 
greater  error  than  the  annual  ridership  calculation  for  the  entire 
system.  For  the  annual  system-wide  ridership  estimate,  there  is  a 
95%  chance  that  the  estimate  is  within  ±10%  of  the  actual  annual 
ridership  based  on  this  approved  sampling  methodology. 

Verification  of  Reported  Performance  During  the 
Audit  Period 

Data  reported  in  the  Service  Standards  Reports  were  compared  to 
that  reported  in  the  Federal  Transit  Administration's  (FTA)  National 
Transit  Database  (NTD).  (All  data  reported  to  the  NTD  are  audited 
regularly  by  the  FTA.)  All  data  were  found  to  be  consistent. 
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I Problems  or  Issues  Identified  During  the  Audit  Period 
I No  significant  problems  or  issues  were  identified. 

I Changes  in  Measurement  Since  the  Audit  Period 

I Subsequent  to  the  audit  period.  Muni  adjusted  the  milestones  to 
I compare  passenger  boardings  to  the  previous  year.  The  milestone 
' for  FY2003  and  FY2004  was: 

• 224  Million  Passenger  Boardings 
Auditor  Recommendations 

The  definition  of  measurement  should  be  amended  to  remove 
the  statement.  Data  will  be  reported  to  the  board  on  a quarterly 
basis." 

I Realistic  goals  for  ridership  should  be  established  based  on  economic 
^conditions  as  well  as  available  service. 
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Proposition  E Municipal  Transportation  Quality  Review 


Fare  revenue  gem 

Goal: 

Piirposo; 

Definition  of  Monsurement: 

jrcitcu  L/jr  

"Tn  inrrRasB  fare  revenue  overall  by  $1.6M  compared  with  FY2DUg 

includino  shares  of  renional  passes  such  as  BART  ^ and  special  event  pas^ ntilrirn  ~ 

Motlind  of  Monsurement: 

r«i,  Inrns  are  collected  electronically  on  board  all  revenue  vehicles  (with  the  exception  ot  Lable  Oar),  utiliz mg  the 
Cubic  Farehox  system.  In  Cable  Car,  a manual  fare  collection  system  along  with  sale  of  special  passes  is  utilize  . 

POP  stations  sell  tickets  at  the  platform.  — , 

MilHStnnos; 

1 Ai  QC  ft/1  avropHori  hY^U02  ~ fSFB  rKVBilUtJ  lllUtdSfi  OT  Vfc.UO  IVI 

FY2001  - fare  revenue  increase  of  $1.96  M exceeueo  sianoaro  rii^uu.: 

exceeded  standard  

From  Mi»i/  SffvJco  StfifK/tin/i  Wf/iorf,  f V 700t  ond  FY2002 


Reported  Performance 

The  cstal)lishc(l  goal  of  increasing  fare  revenue  by  $1.6  M over 
FY2000  fare  revenue  was  accomplished  for  botli  FY200I  and  FY2002. 
Muni  was  unable  to  achieve  the  target  revenue  of  $100  M in  FY2003. 
Table  B-2  summarizes  data  on  fare  revenue  for  FY2000  - FY2003. 


laDie  D-x  M 

Annual  Cash  Fara  Summary 

FY2000 

FY2001 

Improvement 
FY2000  toFY2001 

FY2002 

Improvement 
FY2000  toFY2002' 

' Fms  . 

Improvement 
FY2Q00  to  FY  2003 

A 4 c 1 7 

Motor  Conch 

$15,067,705 

$14,504,449 

$563,256 

$15,949,862 

$882,157 

$14,040,087 

■$l,027,bl/ 

A 4 COO  ♦ oo 

1 rolloy  Coiu'h 

'$13.(181.967 

$13,578,560 

$305,407 

$13,431,373 

$450,594 

$12,249,779 

$1,632,  loo 

A onn 

Li(]ht  Iluil 

$8,642,470 

$9,555,224 

$1,012,754 

$8,254,448 

$288,022 

$7,910,161  ^ 

$■632,309 

43  4 077  A77 

Cnhlo  Cnt 

$12,835,482 

$12,385,518 

$449,964 

$12,385,518' 

$449,964 

$11,008,050 

■$l.o2/,^jz 

I list  Pass  Solos 

$45,427,773 

$47,252,188 

$1,824,415 

$47,537,277 

$2,109,504 

$44,817,894 

$609,879 

43  70n  AHC 

(ItIuH  Moliio 

$5,554,830 

$5,982,166 

$427,536 

$5,731,066 

$176,436 

$6,255,035 

$7UU.^U0 

Ptmitrnnsit 

$784,616 

$782,230 

$17,614 

$865,741 

$101,125 

$1,071,099 

$306,483 

A 4 c cno 

Chmtm  Smvicii 

$15,609 

$15,609 

Totol 

$102,074,643 

$t04,03B,335 

$1,963,692 

$104,155,285 

$2,080,642 

$97,367,714 

■4,400,445 

Mum  SmKr  SuuMi  RfMC-dh  mOOl  m003  and  hi  Qmih  m004 

Daia  In  UaHcs  I»aw  t\ot  b«x*n 


PaoeB*? 


San  Francisco  Municipal  Railway 


Verification  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

Fare  revenue  data  were  collected  by  the  Revenue  Department 
and  reported  monthly  to  the  Finance  Department.  The  Revenue 
Department  provided  information  on  pass  sales  and  sales  of  other 
media  as  well  as  revenue  collected  on  board  vehicles.  The  Revenue 
Department  provided  revenue  by  division,  which  enabled  Finance  to 
allocate  cash  revenue  by  mode. 

The  auditor  was  unable  to  verify  that  the  reports  provided  to  Finance 
accurately  reflected  the  amount  of  revenue  collected,  however  a 
review  of  the  controls  on  revenue  data  showed  a highly  controlled 
and  automated  environment  that  meets  FTA  audit  requirements. 


Venficot/on  of  Reported  Performance  During  the 
Audit  Period 

Based  on  ar>  audit  of  the  monthly  revenue  reports  and  the  calculations 
completed  by  the  Finance  Department,  all  revenue  data  appeared  to 
be  reported  correctly. 

Problems  or  Issues  Identified  During  the  Audit  Period 

Performance  in  FY200I  and  FY2002  were  both  compared  with 
FY2000  and  wore  both  found  to  exceed  the  standard. 

Assuming  each  fiscal  year  was  intended  to  be  compared  to  the  prior 
year’s  performance,  and  not  to  the  FY2000  benchmark,  the  following 
result  is  calculated: 


Annual  Caili  Farn  Summary 


FY  99/00 

FY  00/01 

. , Improvomoni 
FY2000  torY2Q01 

1YOU02:’  1 

„^.lm[iLovomQnl^  ' 
FY2001  Iti‘l'Y2002  . 

Motor  Coach 

$15,067,705 

$14,504,449 

$563,256 

$15,940,662 

$1,445,413 

Trolley  Coach 

$13,881,967 

$13,576,560 

$305,407 

$13,431,373 

$145,187 

Light  Rail 

$8,542,470 

$9,555,224 

$1,012,754 

$8,254,448 

$1,300,776 

Cable  Car 

$12,835,482 

$12,385,518 

$449,964 

$12,386,510 

$0 

Fast  Pass  Sales 

$45,427,773 

$47,252,188 

$1,824,415 

$47,537,277 

$285,080 

Other  Media 

$5,554,630 

$5,982,166 

$427,536 

$5,731,086 

$251,100 

Paratransit 

$764,616 

$782,230 

$17,614 

$865,741 

$83,611 

Total 

$102,074,643 

$104,038,335 

$1,963,692 

$104,155,265 

$116,050 

Percent  Change 

2% 

0.1% 

Proposition  E Municipai  Transportation  Quality  Review  ^ _ 

In  this  case,  the  milestone  of  a $1.6  M increase  in 

exceeded  in  FY200I,  In  FY2002,  the  target  of  an  ^ ° 

M increase  in  fare  revenue  was  ^ 

$1  16,950  increase  in  fare  revenue  between  FY200I  to  FY2UU/. 

Changes  in  Measurement  Since  the  Audit  Period 

Subscciuent  to  tim  audit  period,  Muni  adjusted  the 

fixed  milestone,  rather  than  comparing  fare  revenues  to  the  previo 

year,  Tlic  FY2003  and  FY2004  milestones  were: 

• Achieve  $ 1 00  M in  fare  revenue  in  FY2003 

• Achieve  $ I 1 7 M In  fare  revenue  in  FY2004 

Action  Plan 

Fare  revenue  Is  an  important  management  tool, and  can  be  a surrogate 
for  ridership  .luring  periods  of  fare  stability.  Combined  with  other 
indicators.  It  Is  possible  to  calculate  average  fare  per  passenger,  a 
useful  management  tool  when  looking  at  alternative  fare  structuies. 

Wile  this  indicator  should  be  continued,  the  current  practice  of 
changing  the  benchmark  for  faro  revenue  annually  based  on  the  prior 
year  actual  experience  does  not  allow  for  reasonable  comparison 
across  time.  Rather  than  contimiing  to  alter  the  benchmark,  it  is 
recommendoil  that  the  milestone  be  restated;  "Fare  revenue  should 
increase  by  1 .5%  per  year  compared  with  pi  ior  year  performance." 

Changing  the  milestone  In  this  way  will  allow  for  comparisons  over 
time,  an.l  will  serve  as  a bettor  Indicator  of  the  need  to  adjust  fares 
and  faro  structure. 
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. 3b.  Hours  and  miles  operated  by  mode. 


Goal: 

1 0 increase  hours  and  miles  operated  by  mode  by  1 2% 

Purpose: 

Measure  the  distribution  of  miles  and  hours  traveled  by  mode 

Definition  of  Measurement: 

The  miles  and  hours  that  vehicles  travel  while  in  revenue  service.  Vehicle  revenue  miles  end  roveiuio  hours 
exclude  travel  to  and  from  storage  facilities,  training  operators  prior  to  rovonuo  service,  road  tests  end  dondimnd 
travel,  as  well  as  charter  service. 

Method  of  Measurement: 

Reported  from  schedules,  excluding  any  service  interruptions  or  special  additional  service. 

Milestones: 

1.2% 

From  Mum  Senxe  Standards  Report.  Ff  2001  and  FY2002 


Reported  Performance 

Data  for  hours  and  miles  are  reported  annually,  with  data  from  the 
previous  year  reported  for  the  current  fiscal  year  (i.e..  for  the  FY2002 
Service  Standards  Report,  hours  and  miles  for  FY200 1 was  reported). 
This  is  due  to  the  fact  that  final  mileage  and  hour  calculations  cannot 
be  completed  until  several  months  after  the  conclusion  of  the  fiscal 
yean 


For  both  FY2000  and  FY200 1 .systemwide  mileage  and  liour  increases 
exceeded  the  goal  of  1 .2%  growth  from  the  previous  year.  However, 
this  growth  was  not  distributee!  evenly  among  all  modes.  The  most 
significant  gains  were  for  light  rail.  This  trend  continued  in  FY2002 
and  FY2003.  Table  B-3  summarizes  the  hours  and  miles  and  percent 
change  in  hours  and  miles  for  fiscal  years  1 999, 2000, 200 1 , 2002  and 
2003. 


Table  B-3  Revenue  Hours  and  Miles,  FY 1 999  - FY2003 


% Chang* 

%Chinga 

FY1999 

FY2000 

FY200I 

%QMi«iFY9M0  %Qmii|iFTD00I 

fY2002 

fY2003 

FY0I02 

FY0203 

1 

Hours 

Mi!es 

Hours 

Homs 

V4J 

Hous  * 

Houf; 

V 

fhutt  - 

thuti 

Mihr  :s 

MtiJUIS 

Milm 

liiJUII 

Milut 

h>:ioi  Coach 

1,378.950 

12.388,078 

1.377,052 

12.395,536 

1,405.682 

12.477,997 

001% 

008% 

21% 

07% 

Ur444l$ 

t3.477.575 

t. 881.844 

15.483.730 

0 9% 

7,0% 

7,0% 

15.2% 

•;»  Coach 

1.006,692 

7.027.340 

1,014.636 

7,064.874 

1,015.643 

7,024.834 

oas 

0 5S 

01% 

0 6% 

1.056.197 

7. 78 1. 749 

1.070.3/t 

7.367/59 

4,0iT 

3,7% 

1.3% 

1.2% 

UgMRaiVtM 

426,396 

4,015,172 

474,018 

4,314.232 

518,097 

4.738,100 

11  2% 

74% 

93% 

98% 

571349 

5.483.509 

577.016 

5.53 1.  no 

103% 

1 0,3% 

1.0% 

1,2% 

^Car 

121.044 

494.485 

128.672 

523.642 

128,899 

494.650 

71% 

59% 

r]r.\ 

5 5b 

f 36.591 

441.786 

1 75.3 73 

405091 

0.2% 

10-0% 

'/.0% 

■8,2% 

totai 

2.931,082 

23.925.075 

2.995.378 

24,298.284 

3,018.321 

24,735.581 

27% 

US 

2 4S 

18% 

1307.543 

78.813.508 

3.434.404 

78.707.705 

78% 

7.0% 

3.8% 

0.1% 

Muru  Service  Standards  Rtpon.4th  Quarter  FY200I,FY2002.FY2003  and  1st  Quarter  FY20W 
O&i  >n  ttalto  K*ve  fxH  been  indited 
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Verification  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

Annua,  hours  and  , ni.es  were  caicu»  schedule.  The  nu. 
of  miles  (from  the  Missed  Service 

"'"r "'ditr,  ’ 

„,lL  lo  <lctor...l..c  tho  ncnni  i«.inbor  ol  mite  and  hour  . 

Verification  of  Reported  Performance  During  the 
Audit  Period 

exce  .tiL  of  lire  nu.nher  of  miies  and  hours  reported  for  niotorcoacl 

in  FY200I.  Tho  numher  of  irours  and  ' 

-St  andards  Rodoi  t was  1,405,682  liours  and  '2,477,997  mMes 
numiicr  of  liours  and  miles  reported  to  NTD  w.as  1,405,  28  1 ours 

and  1 2,447,997  miles. (It  seemed  .IS  though  numbers  wcio  mi  . y 

inverted  In  tlie  Service  Standards  Report.) 
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Chonges  in  Meoturement  Since  the  Audit  Period 

ye!r.  The  FY2003  and  FY2004  milestones  were: 

. Revenue  Hours  Goal:  3.6  M hours  in  FY2003  and  3.5  M hours 
in  FY2004 

. Revenue  Miles  Goal:  25  M miles  in  FY2003  and  FY2004 

Action  Plan 

r.^  nrohlpms  identified  with  the  collection  or 
While  ,*'7  that  miles  and  hours  of  service 

that " Umes  of  budge,  constrainu  miles  and  hours  of  service 
Will  continue  to  increase,  or  even  remain  constant. 

Miles  and  hours  of  service  are  important  inpu«  to  the  National 
Transit  Database  reporting,  and  wiil  I-' 

or  not  they  continue  to  be  used  as  a performance  measure.  However, 
it  is  recommended  that  this  NOT  be  continued  as  a service  standard 


Problems  or  Issues  Identified  During  the  Audit  Period 

No  prohloms  or  issues  were  idontified.howevcr  mucit  of  tite  hours  and 
,nlles  d.tt,t  must  bo  t.tbul.tted  maiui.tlly,  whicit  is  quite  time  intensive^ 
Once  tlie  nowTi  apeze  sclioduling  softw.tre  is  implemented,  most  of 
the  calcul.ulons  c.in  he  performed  .nitomatic.tlly,  winch  sltould  save  a 
signiricant  amount  of  stalf  time. 
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4b.  Expenses  incurred  by  mode. 


Goal: 

Expenses  by  Mode  in  each  Fiscal  Year  to  remain  within  budfloted  amount. 

Purpose: 

Measure  the  cost  of  producing  revenue  service  by  mode  os  well  as  budgot  adhoronco. 

Definition  of 
Measurement: 

Expenses  collected  by  mode,  exclude  Worker’s  Comp,  Paralransit  Contract, 
Judgments  and  Claims,  Interfund  transfers,  use  of  Muni  fast  passes  on  BAIU, 
Benefits  for  retirees,  long  term  leaves,  and  services  of  the  City  attorney 

Method  of  Measurement: 

Budget  expenses  are  tracked  by  line  item  and  section  but  are  rolled  up  into  modo 
annually.  Data  reported  on  an  annual  basis. 

Milestones: 

FY2001  - expenses  are  within  budget  FY2002  - expenses  are  within  luidgot 

From  Mum  Service  Standards  Report,  FY  2001  and  FY2002 


Reported  Performance 

The  annual  Service  Standards  Report  simply  reports  whether  costs 
are  within  budget  or  not.  In  both  audit  years,  costs  were  reported 
within  budget. 

Venficat/on  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

Muni’s  expenses  were  allocated  to  expense  codes  in  four  major 
categories: 

• Vehicle  (including  driver  wages); 

• Vehicle  maintenance: 

• Administration;  and 

I • Non  Vehicle  Maintenance  (i.e.,  Facilities). 

I VVithin  each  category,  the  mode  can  be  coded,  to  allow  costs  to 
L be  tracked  by  mode.  While  many  costs  track  directly  to  mode  in 
I a straightforward  manner,  costs,  such  as  most  administrative  costs, 
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must  be  allocated  to  a mode  using  a system  approved  by  FTA  for 
the  National  Transit  Database.  The  methodology,  approved  in  1987, 
allows  Muni  to  allocate  non-mode  specific  costs  to  each  mode 
through  a formula  based  on  the  revenue  miles  by  mode. 

The  measurement  reported  in  the  Service  Standard  Reports  was 
consistent  with  the  NTD  Information  and  is  derived  from  NTD 
reports.  The  actual  measurement  Is  sligluly  different  from  the 
method  described  In  the  table  above.  In  fact,  a correct  statement  of 
measurement  would  be: 

Expenses  collected  and  allocated  by  mode,  exclude  Workers 
Comp,  use  of  Muni  fast  passes  on  BART,  benefits  for  retirees 
not  paid  by  Mum,  and  non-compensoted  long  term  leaves. 

This  definition  Is  consistent  with  the  NTD  report,  and  tlicrcforc 
more  accurately  reflects  what  is  currently  being  reported. 
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Proposition  E Municipal  Transporlation 

Verification  of  Reported  Performance  During  the 
Audit  Period 

The  audit  team  confined  its  investigation  to  the  reporting  done 
by  the  Finance  Department.  The  auditor  was  therefore  unable  to 
comtnent  on  the  accuracy  of  costs  reported  to  each  budget  line  item. 
However;  the  audit  examined  the  process  and  controls  involved  in 
(preparing  the  expenditure  reports  tliat  are  used  in  NTD  and  Service 
Standard  reporting.  All  appropriate  controls  appeared  to  be  in  place 
to  track  costs  l>y  mode  in  an  acceptable  manner. 

Problems  or  Issues  Identified  During  the  Audit  Period 

The  Service  Standard  Report  simply  states  whether  or  not  Muni  was 
"in  budget*'  by  mode.  It  does  not  provide  modal  comparisons,  nor 
docs  It  pi  ovific  lino  item  expenses.  Even  the  information  presented 
Is  a bit  misleading,  since  some  elements  of  the  Muni  budget  are  not 
mode  S[)cclfic,  although  they  arc  ultimately  allocated  by  mode  based 
on  the  NTD  formulas. 


Changes  in  Measurement  Since  the  Audit  Period 

Subsequent  to  the  audit  period,  Muni  has  made  no  changes  to 
the  definition  of  measurement,  method  of  measurement,  nor  the 
milestone. 

Action  Plan 

This  goal  and  the  information  presented  has  limited  value  to  the 
public  or  to  Muni  itself.  In  fact,  Muni  has  no  opportunity  to  exceed 
its  budget  without  receiving  a supplemental  appropriation,  as  a City 
agency  cannot  operate  in  deficit.  Rather  than  comparing  expenses  to 
budget  by  mode,  which  is  an  artificial  comparison,  it  is  recommended 
that  Muni  track  fully  allocated  costs  per  hour  of  service  by  mode. 
This  standard  industry  benchmark  would  allow  Muni  to  compare 
its  unit  costs  against  other  similar  properties  nationwide.  This 
information  is  calculated  by  the  Finance  Department  regularly  for 
the  NTD  report,  and  would  not  require  additional  effort. 

A recommended  performance  measurement  system  would  include. 


Goal: 

Fully  nllocated  unit  cost  per  hour  for  each  mode  will  not  increase  by  more  than  the  Bay  Area  CPI  annually. 

Piiiposn: 

Monsuro  the  cost  of  producing  revenue  service  by  mode  as  well  as  the  ability  to  control  unit  costs. 

Definition  of 
Mnnsmomont: 

Expenses  collected  and  allocated  by  mode  will  be  based  on  the  NTD  methodology,  (exclude  Worker's  Comp,  use  of  Muni 
fast  passes  on  BART,  benefits  for  retirees  not  paid  by  Muni,  and  non-compensated  long  term  leaves)  divided  by  the 
number  of  hours  operated  by  mode  to  calculate  unit  costs  per  hour. 

Mothmi  of  Mnnsuromnnt: 

Expenses  by  mode  are  calculated  using  an  NTD  approved  methodology  on  an  annual  basis.  Annual  hours  by  mode  are 
calculated  using  an  NTD  approved  methodology  on  an  annual  basis. 

Milestones; 

Costs  per  hour  of  service  for  each  mode  shall  not  exceed  the  Bay  Area  CPI  in  each  year. 
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C,  Staffing  Performance 

The  Proposition  E Service  Standards  for  staffing  performance  focus 
on  staffing  levels  and  the  ability  to  retain  new  employees  They 

Ic.  Net  vacancies  by  position  (vacancies  remaining  once 
promotions  and  new  hires  have  been  deducted  from  retirees 
or  resignations)  for  each  division. 

2c.  Attrition  rates  for  new  employees,  by  division  and  level. 

Key  Recommendations 

The  following  are  key  recommendations  based  on  the  findings  of  our 
audit  of  FY200I  and  FY2002  Service  Standards  data  and  on  trends 
observed  in  unaudited  FY2003  and  FY2004  Service  Standards  data: 

For  Service  Standard  I c,  the  great  majority  of  vacancies  cannot 
be  filled  due  to  budgetary  policies.  In  order  to  ensure  that 
service  provision  does  not  suffer,  Muni  has  focused  on  making 
sure  that  maintenance  and  operator  positions  are  filled.  Con- 
sequently, the  auditor  recommends  that  this  Service  Sundard 
be  limited  to  Transit  Operators  and  Maintenance  personnel, 
or  that  the  goals  for  other  divisions  (exclusive  of  Transit  Op- 
erators and  Maintenance  personnel)  be  revised  so  that  they 
can  be  reasonably  met  given  budgetary  constraints. 
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In  order  to  track  the  attrition  rate  for  now  hires  in  Service 
Standard  2c,  attrition  rates  should  either  bo  reported  at  the 
end  of  now  employees'  probationary  periods  or  within  one 
year  after  hire.  To  calculate  attrition,  only  those  employees 
who  separate  after  six  months  or  one  year  of  service  should 
be  counted.  Furthermore,  there  Is  no  information  provided  in 
this  measure  about  why  employees  are  voluntarily  separating 
from  Mum  service.  If  It  Is  possible  to  obtain  this  information 
from  the  Position  Control  System  database,  then  it  should 
also  be  summarized  and  Included  In  this  measurement. 
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/c.  Net  vacancies  by  position  (vacancies  remaining  once  promotions  and  new  hires 
have  been  deducted  from  retirees  or  resignations)  for  each  division. 


Gnni' 

No  (ireater  than  5%  vacancy  rate.  

Piirposo! 

Ffficionev  IbvbI  of  tho  dopartmEnt  in  hiring. 

Definition  of  Monsiiromont: 

Mnnihiv  mBasurcniBnt  of  not  vacanciBS  against  budgetad  positions  by  division. 

Mothnd  of  Monsiiromont: 

TliP  Vacancy  Renorl  will  be  the  basis  of  the  data  reported  to  the  Board  on  a quarterly  basis. 

Milostonos: 

5% 

Troin  AI1//1/  SrF  Wfp  Suindofilt  Ap/jorf,  FY  2001  and  FY2002 


Reported  Performance 

The  luiinher  of  budgeted  positions  and  the  number  and  percentage 
of  vacancies  at  the  beginning  and  end  of  each  quarter  were  reported. 
Table  C-l  shows  the  percentage  of  scheduled  positions  that  were 
vacant  at  tUc  end  of  the  fourth  quarter  for  fiscal  years  2001, 2002 
and  2003  as  well  as  at  the  end  of  the  first  quarter  of  FY2004.  Overall. 
Muni  mot  its  goal  of  having  a vacancy  rate  of  less  than  5%  for  all  four 
years.  The  vacancy  rate  for  transit  operators  was  the  lowest,  while 
that  for  finance  and  administration.  GM  Programs/Safety  andTraining 
was  highest. 

Tabic  C- 1 Vacancy  Rate,  FY200  I - FY2004 
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Verification  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

The  Annual  Salary  Ordinance  was  used  to  determine  the  number  of 
budgeted  positions  for  each  fiscal  year."Vacancies"  were  any  type  of 
permanent  separation,  including  releases,  dismissals,  retirements  and 
transfers.  Leaves  were  not  counted  as  vacancies.  Vacant  positions 
were  tabulated  using  the  Position  Control  System  database.  The 
database  was  queried  for  all  vacancies,  since  it  cannot  create  separate 
vacancy  queries  for  each  department.  (This  was  done  manually.) 
Staff  checked  to  ensure  that  vacancies  tabulated  for  the  end  of  the 
previous  quarter  matched  the  number  of  vacancies  reported  for  the 
beginning  of  the  subsequent  quarter. 

Verification  of  Reported  Performance  During  the 
Audit  Period 

Based  on  an  audit  of  data  for  the  third  quarter  of  FY2002.  vacancy 
rates  appear  to  have  been  calculated  and  reported  correctly. 

Problems  or  Issues  Identified  During  the  Audit  Period 

If  the  Position  Control  System  database  could  be  queried  to  report 
the  number  of  vacancies  by  department,  it  would  save  a significant 
amount  of  staff  time. 
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Chonges  in  Alcasurcment  Since  the  Audit  Period 

Subsequent  to  the  audit  period,  Muni  has  made  no  changes  to 
the  definition  of  measurement,  method  of  measurement,  nor  the 
milestone. 

Auditor  Recommendations 

The  great  majority  of  vacancies  cannot  he  filled  due  to  budgetary 
policies.  In  order  to  ensure  tliat  service  provision  docs  not  suffer, 
Muni  lias  focused  on  making  sure  that  maintenance  and  operator 
positions  arc  filled.  Consequently,  the  auditor  recommends  that  this 
Service  Standard  bo  limited  to  Transit  Operators  and  Maintenance 
personnel,  or  that  tlie  goals  for  other  divisions  (exclusive  of  Transit 
Operators  and  Maintenance  personnel)  be  revised  so  that  they  can 
be  reasonably  met  given  budgetary  constraints. 
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2c.  Attrition  rates  for  new  employees,  by  division  and  level. 


Gonb 

No  (ircater  than  10%  ■ ■ — 

Plirposo: 

MBasuromenl  of  effectiveness  of  recruitment  and  employee  satisfaction  by  the  rate  of  voluntary 
scnaralions  for  new  employees. 

Dofinitinn  uf  Monsiiromont: 

Number  of  employees  by  division  and  class  who  are  released  during  probationary  period  or  who 
are  voluntarily  separated  during  probation.  Will  break  down  attrition  within  probationary  period, 
as  well  as  attrition  one  year  after  liire.  Data  will  be  reported  to  the  board  on  a quarterly  basis. 

Mfithnil  (if  Munsiirnmont; 

Vacancy  Report  will  provide  data  for  quarterly  reporting. 

Milnstnmis: 

10% 

Reported  Performance 

Eacli  quarter,  tlic  nunibor  of  new  hires,  releases,  and  voluntary 
so()arailons  that  occur  within  each  division  were  reported.  The 
attrition  rate  was  calculated  by  dividing  the  total  number  of  releases 
and  voluntary  separations  that  occur  within  a quarter  by  the  total 
number  of  new  hires  within  that  quarter.  During  the  two  fiscal 


years  audited.  Muni  never  achieved  an  attrition  rate  lower  than  1 1% 
(which  occurred  during  the  first  quarter  of  FY200I).  Attrition  rates 
remained  high  in  FY2003  and  FY2004,  as  shown  in  Table  C-2. 


Tabic  C-2  Attrition  Rate.  FY200 1 - FY2004 
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The  attrition  rates  reported  in  the  Service  Standards  Reports  do 
not  accurately  represent  actual  attrition  rates  for  new  employees, 
however.  The  rate  was  based  on  the  number  of  people  who  were 
hired  or  who  separated  during  a quarter,  however  the  employees 
who  separated  during  a quarter  were  not  necessarily  the  same  as 
those  who  were  hired  during  that  quarter.  Furthermore,  voluntary 
separations  included  anyone  who  voluntarily  separated,  regardless 
of  their  length  of  employment.  This  issue  is  discussed  further  in  the 
subsequent  sections. 

Verification  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

The  Position  Control  System  database  was  used  to  determine  the 
number  of  new  hires,  releases  and  voluntary  separations  within  each 
quarter.  Releases  were  counted  as  those  staff  who  were  released 
for  disciplinary  or  non-disciplinary  reasons  during  their  probationary 
period  (typically  the  first  six  months  of  employment).  Voluntary 
separations  included  staff  who  either  quit  work  or  transferred.  It 
did  not  include  retirements.  The  number  of  voluntary  separations 
reported  in  a given  quarter  were  for  anyone  who  left  during  that 
quarter,  regardless  of  the  length  of  his  or  her  employment. 

Verification  of  Reported  Performance  During  the 
Audit  Period 

Based  on  an  audit  of  data  for  the  third  quarter  of  FY200I,  attrition 
rates  were  not  calculated  correctly.  Voluntary  separations  included 
all  employees  who  separated  from  Muni  during  a particular  quarter, 
regardless  of  their  length  of  employment. 
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Problems  or  Issues  Identified  During  the  Audit  Period 

Attrition  rates  were  not  reported  for  both  the  probationary  period 
and  for  one  year  after  hire,  per  the  stated  definition  of  measurement. 
Either  data  should  bo  tabulated  differently,  or  the  stated  definition 
of  measurement  should  be  amended  as  follows:  "Will  break  down 
attrition  within  probationary  period." 

The  reporting  methods  used  during  FY200I  and  FY2002  do  not 
provide  information  on  the  attrition  rates  for  new  employees.  The 
employees  who  were  hired  within  a [)articular  quarter  were  not 
necessarily  the  same  employees  who  separated  within  that  same 
quarter. 

Changes  in  Measurement  Since  the  Audit  Period 

Subsequent  to  tfie  audit  period,  Muni  has  made  no  changes  to 
the  definition  of  measurement,  method  of  measurement,  nor  the 
milestone. 

Auditor  Recommendations 

In  order  to  track  the  attrition  rate  for  now  hires,  attrition  rates 
should  cither  be  reported  at  the  end  of  new  employees*  probationary 
periods  or  within  one  year  after  liirc.  To  calculate  attrition, only  those 
employees  who  separate  after  six  months  or  one  year  of  service 
should  be  counted.  Furthermore,  there  Is  no  information  provided 
in  this  measure  about  why  employees  arc  voluntarily  separating  from 
Muni  service.  If  It  Is  possible  to  obtain  this  Information  from  the 
Position  Control  System  database,  then  it  should  also  be  summarized 
and  included  in  this  measurement. 
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D.  Customer  Service 

Customer  service  addresses  passenger  safety  on  board  vehicles  and 
at  stops;  the  ease  with  which  passengers  can  navigate  and  use  the 
transit  system  to  get  where  they  need  to  go  in  a timely  and  efficient 
manner;  the  availability  of  information  about  schedules,  delays  or 
other  service  interruptions;  and  the  responsiveness  of  transit  agency 
staff  when  customer  problems  or  complaints  arise.  Pioposition 
E includes  tlie  following  Service  Standards  aimed  at  measuring 
customer  service: 

• Id.  Development  of  an  annual  marketing  plan  identifying 
specific  programs  and  [)rojccts  that  will  promote  increased 
patronage. 

• 2d.  Publication  and  distribution  to  the  public  of  schedules  for 
all  trips  taken  by  all  vehicles  that  shall  consists  of  specific  ar- 
rival limes  at  terminals  and  established  intermediate  points. 

• 3d.  Operator  conduct  complaints. 

• 4d.  Annual  passenger  surveys  and  follow-up  by  manage- 
ment. 

• 5d.  improvements  In  public  information  regarding  vehicle 
delays  during  operations  as  well  as  any  general  user  infor- 
mation regarding  system  modifications,  route  changes,  and 
schedules. 

• 6d.  Efforts  CO  Improve  driver  training,  technical  as  well  as  ac- 
citlent  lollow-up. 

• 7d,  Number  of  crime  incidents  on  Municipal  Railway  vehicles 
or  in  Municipal  Railway  facilities. 
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Key  Recommendations 

The  following  are  key  recommendations  based  on  the  findings  of  our 
audit  of  FY200I  and  FY2002  Service  Standards  data  and  on  trends 
observed  in  unaudited  FY2003  and  FY2004  Service  Standards  data: 

• For  Service  Standard  1 d,  marketing  priorities  should  be  made 
within  budget  constraints  and  with  the  goal  of  increasing 
patronage:  data  from  ridership  surveys  should  be  used  to 
evaluate  the  success  of  marketing  efforts. 

• For  Service  Standard  2d.  the  current  process  of  schedule 
preparation  is  cumbersome,  time  consuming,  and  relies  on 
the  background  and  knowledge  of  one  key  staff  person.  The 
capabilities  of  the  new  Trapeze  software  should  be  fully  utilized 
in  order  to  simplify  and  streamline  the  process  of  schedule 
preparation. 

• For  Service  Standard  3d,  operator  conduct  complaints  need 
to  be  more  clearly  measured  and  reported,  realistic  goals  for 
reducing  the  number  of  operator  conduct  complaints  should 
be  established,  and  the  efficiency  and  accuracy  of  complaint 
information  uptake  should  be  improved. 
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• To  be  truly  useful,  Service  Standard  5d  should  be  divided  into  two  goals  to  distinguisl)  between  routine  information  provided  in  real 
time  and  longer-term  information  provided  as  part  of  service  or  schedule  change  and/or  construction  projects.  To  further  differentiate 
between  immediate  information  provided  to  passengers  on  the  system  during  delays  or  other  unusual  events,  and  longer-term  infor- 
mation provided  to  address  construction  or  other  longer-term  events,  the  following  systems  of  measui  cmcnt  arc  recommended: 


Goal: 

Improve  Real-Time  Passenger  Information  rogardino  service  delays. 

Purpose: 

Provide  the  public  with  information  regarding  service  delays  that  will  enable  them  to  make  choices  nhout  their  trip. 

Definition  of  Measurement: 

Improve  passenger  information  by  communication  of  service  problems  and  other  information  to  each  vehicle,  the  station 
platforms,  the  Telephone  information  Center,  media,  the  Service  Holline,  and  Service  Quolity  Teams. 

Operators  will  make  passenger  announcements  in  all  cases  whore  service  delays  exceed  S mimitos  in  the  subway  iiiul  lU 
minutes  on  surface  lines. 

Central  Control  will  make  both  visual  and  verbal  announcements  in  all  stations  whimi  service  delays  oxcueil  3 miiuites  on  all 
LRV  lines. 

Service  Qualilv  Teams  will  be  assigned  to  all  unscheduled  service  disruptions  oxceodiiui  two  hours. 

Method  of  Measurement: 

Central  Control  logs. 

Passenger  Satisfaction  Surveys.  — 

Milestones: 

95%  of  all  delays  of  3 minulos  or  moie  lopoilcd  via  slalion  address  syslum. 

Service  Quality  Teams  assigned  to  all  unscliodulod  soivico  disiuptions  uKCOuiling  2 liouis. 

75%  of  all  passengers  returning  surveys  indicate  that  inlormatioii  rugarding  sorvicii  delays  is  nilutpialii  mid  iisoliil. 

In  order  to  meet  the  accident  reduction  goals  esublishcd  in  Service  Sundard  6d.  it  will  be  necessary  to  develop  a comprehensive 
approach  and  coordinated  effort  to  reduce  accidents  across  divisions  and  departments.  Driver  training  was  fouru  to  he  s^'^lcn  . 
Therefore  the  focus  of  accident  reduction  efforts  should  be  enforcement  of  safety  rules  and  procedures  and  accident  prevention 


through  thorough  review  of  all  accidents. 
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Id.  Development  of  annual  marketing  plan  identifying  specific  programs  and  projects  thot 


will  promote  increased  patronage. 


Goal: 


Piirposo: 


Dofinition  of  Monsiiromcnt: 


otiiod  of  Monsiirnmont: 


Milostonos: 


Develop  Marketino  Plan  liy  January  1,2001 


T„  r . enno.v  of  markelinn  tools  that  will  provide  the  public  with  an  incentive  to  utilize  the  services  of  Mm 


Marketing  plan  developetl. 


Marketing  Plan  conipleterl  and  approved  (or  implementation. 


FY200I  - Marketing  Plan  published  FY2002  - Marketing  Plan  published 


From  Mtinl  SpfvUo  Staiulatdi  Hcl)nnJY  2001  and  fY7002 


Reported  Performance  During  the  Audit  Period 

M.ti  kctlnp,  I’l.ins  wore  puhlislicd  in  botli  FY200I  and  FY2002. 

Verification  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

Muni's  Mai  kciinp,  Director  provided  copies  of  the  FY200I  and 
FY2002  Marketing  Plans  as  well  as  samples  of  marketing  materials 
that  were  produced. 

Verification  of  Reported  Performance  During  the 
Audit  Period 

Marketing  Plans  wore  |Hihllshed  in  both  FY200I  and  FY2002. 


Problems  or  Issues  Identified  During  the  Audit  Period 

Overall,  marketing  plans  were  too  ambitious  for  the  budget  provided. 
Moi'e  strategic  goals  and  objectives  should  be  set,  including  some 
method  of  feedback  or  evaluation  to  know  that  the  marketing  actions 
are  achieving  the  expected  results.  For  example,  targeted  programs 
for  off-peak  riders  should  be  measured  by  ridership  changes  after 
program  implementation. 

Changes  in  /Vleasurement  Since  the  Audit  Period 

Subsequent  to  the  audit  period,  Muni  has  continued  to  set  a January 
I milestone  for  completion  and  approval  for  implementation  of  the 
Marketing  Plan. 
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Action  Plan 

Budget  and  policy  planning  could  be  better  coordinated  so  that  the 
most  cost  effective  and  most  essential  marketing  activities  are  funded. 
The  marketing  plan  identifies  activities  that  should  be  undertaken, 
but  where  there  is  insufficient  budget,  the  process  of  priority  setting 
becomes  ad  hoc  or  driven  by  the  opportunity  for  outside  funding. 
(Note:  Significant  improvements  have  been  made  in  this  area  since 
the  completion  of  the  audit  period  and  may  be  further  investigated 
in  the  subsequent  cask  to  develop  action  plans.) 

Key  problems  and  actions  include: 

• Setting  priorities  for  marketing  efforts  - The 
marketing  plan  sets  out  objectives  for  improving  Muni’s  image 
and  developing  ridership  in  targeted  markets.  Staff  has  been 
very  creative  in  developing  and  implementing  initiatives  to 
date,  but  with  a constrained  budget,  staff  should  set  priorities 
for  those  initiatives  chat  best  achieve  the  goal  of  increased 
patronage. 

• Utilizing  data  from  ridership  surveys  to  evaluate 
success  -There  is  no  quantitative  measurement  established 
for  this  performance  goal;  however,  data  from  ridership  surveys 
should  be  used  to  determine  whether  focused  marketing  is 
creating  the  changes  in  ridership  anticipated.  For  example,  if 
the  objective  is  to  increase  off-peak  ridership  (on  certain  lines, 
line  segments,  or  systemwide)  then  operations  data  collected 
could  indicate  the  change,  and  the  ridership  survey  provide 
the  reasons  for  chat  change. 
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2d.  Publication  and  distribution  to  the  public  of  scheduler  for  all  '’'f 

vehicles  that  shall  consist  of  specif, c arrival  times  at  terminals  and  established 


intermediate  points. 


Publish  a conmlete  liniatable  by  January  1 , 2UU 1 

PiirnnKn* 

PrnvidB  riilGfs  with  an  updated  schedule.  — — 

Dufinitinii  of  M«nsur«ni(jnt: 

Publication  and  distribution  of  schedules  (or  all  trips  taken  by  all  vehicles  that  shall  consist  of  specific  arrival  times 
at  tnrminals  and  oslablished  intermediate  points.  

MfMliod  nf  Monsiirnmunt: 

Distribution  of  the  timetable  to  the  public.  Muni  is  in  the  process  of  reviewing  the  schedules  ot  all  the  bnes  Once 
tlio  review  is  complete,  we  will  publish  schedules  for  individual  lines,  as  well  as  an  updated  system  wide  schedule. 

Milfjstnnns: 

rY2nni  - Printinn  and  distribution  of  timetable  FY2002  - Printing  and  distribution  ot  timetable 

r I oni  Muni  Spfvho  Stufuldtdt  ftp/io;  f,  FY  ?00l  aiuJ  FY2002 


Reported  Performance  During  the  Audit  Period 

Miinl  i>iil)lislio(l  ;wm\  distrilnitcd  a complotc  timetable  for  2001  on 
Auj'.usi  10.  2001,  and  a complotc  timetable  for  2002  on  August  31, 
2002. 

Vehfient/on  of  Dcfi/iition  and  Method  of 
Measurement  During  the  Audit  Period 

Scl)odulos  wore  dovolo|)ctl  from  tlio  database  of  tlio  actual  runs 
liid  on  in  the  Scheduling  Department.  When  a new  driver  sign-up 
occurred,  the  Scheduling  Department  developed  the  run  cuts  and 
assignments.  This  information  was  then  sent  to  the  Information 
Technology  De[)artmeiu  where  staff  converted  the  data  into  an 
Excel  spreadsheet  file.  This  Excel  file  was  then  sent  to  Planning 
Division  staff,  who  reviewed  and  conferred  witli  Scheduling  about 
any  changes.  Schedule  changes  were  not  flagged  automatically; 
recognition  of  changes  depended  on  the  staff  person’s  knowledge  of 
the  system.  Materials  were  edited  and  then  sent  to  the  publisher. 


Verification  of  Reported  Performance  During  the 
Audit  Period 

Two  printed  timetables  have  been  published  during  the  audit  period. 
The  first,  published  on  August  1 8. 200 1 .was  completed  in  a very  short 
time  with  the  extensive  use  of  interns.  The  second  was  published 
on  August  3 1 . 2002.  The  timetables  were  distributed  in  buses,  Metro 
stations.  City  Hall  (Info  Desk  and  theTreasurer’s  Office)  and  the  Main 
Library.  Staff  also  filled  mail  order  requests,  and  the  information  was 
made  available  on  the  Muni  and  regional  transportation  websites. 

Prob/ems  or  Issues  Identified  During  the  Audit  Period 

With  the  full  implementation  of  the  new  Trapeze  software,  staff  will 
have  the  ability  to  overlay  new  schedules  on  existing  ones  for  a 
quicker  and  more  efficient  understanding  of  any  changes.  Currently, 
the  process  is  cumbersome,  time-consuming  and  reliant  upon  the 
background  and  knowledge  of  one  key  staff  person. 


lyilljjilll  San  Francisco 


Municipal  Railway 


No  new  timetable  is  expected  to  be  published  for  2003  due  to 
the  high  expense.  Staff  is  identifying  alternative  media  for  reaching 
riders. 


Chonges  in  Measurement  Since  the  Audit  Period 

Subsequent  to  the  audit  period.  Muni  adjusted  the  milestones  to 
timetable  publications  to  the  previous  year.  The  FY2003  and  FY2004 
milestones  were/are: 

• FY2003  - Publish  timetable 

• FY2004  - Publish  a complete  timetable  during  FY2004 

Action  Plan 

The  auditor  identified  schedule  preparation  timeliness  and  efficiency 
as  a key  issue  for  this  Service  Standard.  The  current  process  for 
preparation  of  the  schedules  is  cumbersome,  time-consuming 
and  reliant  upon  the  background  and  knowledge  of  one  key  staff 
person.  Timely  response  to  changes,  for  example,  those  resulting 
from  the  proposed  FY2004/2005  budget,  is  difficult.  Ultimately,  full 
implementation  of  the  Trapeze  software  will  allow  staff  to  overlay 
new  schedules  on  existing  ones  for  a quicker  and  more  efficient 
understanding  of  the  changes  and  should  allow  quicker  response. 
However,  staff  must  be  adequately  trained  in  the  use  of  the  software, 
or  work  practices  will  continue  as  they  are  at  present. 
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3d. 


Trom  Mi»il  SofWrr  Sto(K/(JK/*  fY  2001  and  FY2002 

Reported  Performance 

In  FY200I,  90%  of  Opcintor  Complaints  were  resolved  within 
the  30-day  timeframe,  meeting  tlie  stated  goal  of  75%  resolution 
witliin  30  days.  In  FY2002,  only  70%  were  resolved  within  the 
timeframe,  and  the  75%  goal  was  not  met.  Performance  improved 
in  FY2003,  and  the  75%  goal  was  achieved.  Table  D-l  shows  the 
percentage  of  operator  complaints  resolved  within  30  days  (the 
number  of  complaints  closed  within  the  timeframe  divided  by  the 
number  of  complaints  recommended  for  neutral  hearings)  for 
FY200I  - FY200d. 

T.iblo  D-l  Percent  of  Operator  Complaints  Resolved  in  30  Days,  FY200 1 - FY2004 


FY2001 

FY2002 

FY2003 

1st  Quarter 
FY2004 

//  ol  opiiiiiUir  ciimptaints  roquitiiin  follow  up 

733 

645 

479 

337 

II  rocomnimulod  (m  mnitrnl  honrinps 

303 

284 

124 

62 

H closmi  within  tiui  timo  liomo 

272 

199 

97 

35 

% closmi  within  tho  tinm  Irnmo 

90% 

70^ 

78% 

57% 

Fiolu  AIumI  Scivfif  QiMitff  fY200l,  FY2002,  FY2003  and  1st  QiMftcf  rY2004 


D.u.\  In  itAllcs  hAvt>  not  l'rt>n 


Operator  conduct  complaints. 


: compiainis.  — 

be  .s.l..d « 30  d.,»7^0%  .f 

HDpOflS  aillluaMy. 


Icfiliitinn  nf  Monsiiroiiicnt: 
/lotlinil  of  Monsiircnicnt: 


lilnstnnos; 
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As  shown  in  Table  D-2.  total  complaints  were  reduced  by  5% 
between  FY2000  and  FYI999.  However.  Muni  did  not  attain  its  goal 
of  a 10%  reduction  in  complaints  in  FY200I,FY2002,  or  FY2003.  In 
FY2001,  the  number  of  complaints  exceeded  the  goal  by  2.454.  In 
FY2002.  the  number  of  complaints  exceeded  the  goal  by  1 .733.  This 
trend  continued  in  FY2003,  when  the  number  of  actual  complaints 
exceeded  that  goal  by  704. 


Table  D-2  Total  Number  of  Passenger  Service  Reports,  FY 1 999  - FY2004 


Tarnpt 

FY99 

Actual 

FY  00 
Actual 

FY01 

Actual 

FY01 

Goal 

FY02 

Actual 

FY  02 
Goal 

FY03 

Actual 

FY03 

Goal 

IstQFYlM 

Actual 

IstQfYUn 

Goal 

Reduce  PSR's  by  10% 

12,456' 

11,674 

12,931 

10,507 

13,373 

11,640 

12.740 

12,036 

711 

704 

From  Mum  Serv»ce  Storxtonfs  ftfpoft.4ih  Quontr  mOOt.  fY2002.  fT2003  omi  lit  QuofW  m004 
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Verification  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

A flctailof]  process  manual  and  flow  chart  dictated  how  complaints 
were  recorded  and  resolved.  Data  were  collected  and  analyzed 
in  accordaficc  witli  tiiis  manual  and  the  described  definition  and 
method  of  measurement. 

Verification  of  Reported  Performance  During  the 
Audit  Period 

Based  oti  an  audit  of  data  from  FY200I  and  the  first  through  third 
(Hiartcrs  of  FY2002,  ()crformance  appears  to  have  been  reported 
correctly. 

Problems  or  Issues  Identified  During  the  Audit  Period 

In  FY2002,  approximately  50%  of  driver  complaints  referred  for 
administrative  liearing  were  drop[)od  because  of  failure  of  the 
complainant  to  attend  the  hearing.  Because  of  tins  situation,  a new 
procedure  for  handling  driver  complaints  wont  into  effect  on  July 
I,  2003.  The  major  change  was  at  the  Division  level,  where  instead 
of  automatic  administrative  iiearings,  there  is  a now  a consultation 
l)ctwecn  management  and  labor  as  to  tlie  appropriate  action,  with 
tl)o  goal  of  empowering  management  and  looking  for  ways  to  retrain 
and  su[)port  drivers. 

In  both  FY2001  and  FY2002,  Muni  was  unable  to  meet  its  goals  for 
reducing  the  number  of  Passenger  Service  Reports.  This  will  be 
investigated  further  in  the  subsequent  task  to  develop  action  plans 
to  improve  performance. 


Changes  in  Measurement  Since  the  Audit  Period 

Subsequent  to  the  audit  period,  Muni  has  made  no  changes  to 
the  definition  of  measurement,  method  of  measurement,  nor  the 
milestones. 

Action  Plan 

This  action  plan  focuses  on  three  key  issues  identified  by  the 
auditor: 

• Clarity  on  what  performance  is  being  measured 

• A need  for  realistic  goals 

• Efficiency  and  accuracy  of  complaint  information  intake 

Clarity  on  what  Performance  is  Being  Measured 

The  operator  conduct  data  is  reported  through  the  Passenger  Service 
Report  (PSR)  database.  The  service  standard  is  specific  to  operator 
conduct,  not  the  whole  universe  of  PSR’s.  Measurement  should  focus 
on  the  nature  and  disposition  of  the  operator  conduct  complaints: 
the  FY2004  Service  Standard  Reports  are  now  more  clearly  and 
appropriately  focused.  However,  the  metric  reported  is  still  total 
PSR's,  not  operator  conduct  complaints.  This  should  be  revised 
to  properly  reflect  the  intent  of  the  performance  standard.  Staff 
should  also  clarify  in  the  decision  flow  chart  the  criteria  and  process 
through  which  the  intake  numbers  on  operator  conduct  complaints 
are  reduced  to  non-actionable  items,  since  a large  percentage  do 
not  go  forward  due  to  such  problems  as  lack  of  information,  no 
customer  identification,  or  the  described  action  is  not  considered 
an  infraction  of  Muni  rules.  The  auditor  also  recommends  deleting 
commendations  and  station  ops  from  the  totals,  since  they  do  not 
have  to  do  with  complaints  about  operator  behavior  and  confuse 
the  understanding  of  the  goal  for  reduction. 


San  Francisco  Municipal  Railway 

It  makes  sense  to  have  the  30-day  standard  apply  only  to  the  actionable 
operator  complaints,  with  the  assumption  that  ait  other  operator 
conduct  complaints  have  been  suitably  disposed  of.  A reporting 
format  that  shows  total  actionable  items  pending  and  length  of  time 
each  quarter  would  be  more  useful  than  just  showing  totals.  It  is 
not  clear  to  the  reader  whether  the  quarterly  totals  are  the  number 
received  or  the  number  pending.  The  information  system  should 
be  able  to  generate  a report  at  any  time  (not  just  quarterly)  that 
shows  total  pending  operator  conduct  complaints  and  time  lapsed 
for  each. 

A Need  for  Realistic  Goals 

The  continued  10%  annual  reduction  in  PSRs  is  not  a realistic  goal 
without  some  kind  of  qualifier  such  as  a multi-year  timeframe  or 
target  threshold.  The  goal  needs  to  address  the  following  question: 
What  is  the  acceptable  level  of  operator  conduct  complaints  in 
the  organization?  In  other  words,  what  would  represent  not  just 
improvement,  but  success?  One  suggestion  is  to  look  at  the  ultimate 
target  in  relationship  to  other  related  goals,  such  as  operator  training 
or  service  reliability.  Since,  at  Muni,  there  is  a historical  relationship 
between  service  reliability  and  operator  conduct  complaints,  a 
realistic  target  might  be  to  reduce  complaints  to  a certain  level  and 
then  not  have  them  exceed  service  reliability  indices. 

In  setting  a base  target,  data  from  other  systems  could  give  an 
indication  of  some  appropriate  levels.  For  example.  Sacramento  has 
an  average  of  3,000  total  PSR*s  (approximately  66%  of  these  are 
operator  conduct  complaints)  quarterly  and  a daily  ridership  base 
of  100,000.  Muni  needs  to  determine  what  the  acceptable  target 
number  of  total  PSR’s  should  be  and  the  percentage  of  those  PSR*s 
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that  are  attributable  to  operator  conduct  given  Muni’s  much  more 
congested  environment  and  a daily  ridership  of  over  700.000.  The 
understanding  is  that  these  goals  work  together,  in  that  reducing 
resolution  time  will  ultimately  result  in  changed  operator  behavior 
and  fewer  complaints: however,  expecting  a 10%  reduction  indefinitely 
from  year  to  year  is  not  a realistic  goal. 

Efficiency  and  Accuracy  of  Complaini  Information  Intake 

The  majority  of  the  operator  conduct  complaints  arc  recorded  by 
hand  during  telephone  conversations.  Other  complaints  are  received 
by  written/mailed  or  email  correspondence.  Personnel  who  take 
the  information  should  be  proficient  in  data  Input  directly  onto  the 
electronic  form  as  it  Is  communicated  to  tliem.  This  would  greatly 
improve  efficiency  and  reduce  the  chance  of  erroi*.  If  personnel  do 
not  have  the  skills  to  perform  these  functions,  ti  aining  should  be 
provided.  Improved  intake  could  also  affect  the  number  of  complaints 
that  arc  actionable  (require  follow-through). 
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Reported  Performance  During  the  Audit  Period 

Results  of  the  FY200I  Rider  survey  were  presented  to  the  MTA 
Board  on  August  7,  2001,  following  presentation  to  the  Citizens 
Advisory  Committee  (CAC).  Another  Rider  survey  was  conducted 
in  July  2002.  Employee  surveys  were  conducted  in  June  2001  and 
June  2003.  Focus  group  meetings,  as  stated  in  the  Definition  of 
Mcasuromciu,  wore  not  held. 

Verification  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

The  rider  and  employee  surveys  were  conducted  by  the  firm  of 
David  Biiuler  Research  under  contract  to  Muni.  Four  hundred 
respondents  Aged  eighteen  or  older  who  rode  Muni  in  the  past  six 
months  wore  interviewed  on  June  1st  - 3rd,  2001  and  July  21st  - 
24tli,  2002.  These  riders  were  randomly  selected  residents  of  the 
City  of  San  Francisco  with  working  telephone  numbers  that  met  the 
aforementioned  criteria.  Results  for  both  surveys  were  subject  to  a 
sampling  error  of  4.9%. 


In  June  200 1 . employee  survey  results  were  obtained  from  921  self- 
administered  questionnaires  from  employees  in  ten  departments, 
with  the  majority  coming  from  Transportation  and  Maintenance. 

In  June  2002,  results  were  obtained  from  504  self-administered  j 
questionnaires  from  employees  in  1 2 departments,  with  the  majority  j 
again  coming  from  Transportation  and  Maintenance. 

Verification  of  Reported  Performance  During  the  Audit 
Period 

Rider  Surveys  were  conducted  in  June  200 1 and  July  2002.  Employee 
surveys  were  conducted  in  June  2001  and  June  2002. 


San  Francisco  Municipal  Railway 

Problems  or  Issues  Identified  During  the  Audit  Period 

Completion  of  the  surveys  and  communication  of  the  information 
to  the  Board  accomplishes  the  first  part  of  the  goal.  However,  the 
connection  between  findings  and  recommended  actions  for  future 
marketing  and  service  planning  initiatives  has  not  been  documented. 
In  addition,  no  protocols  have  been  established  for  trend  or  market 
segment  analysis. 

Chonges  in  /Vleasurement  Since  the  Audit  Period 

Subsequent  to  the  audit  period.  Muni  has  made  no  changes  to  the 
definition  of  measurement,  method  of  measurement,  or  milestones 
for  annual  surveys. 

Auditor  Recommendations 

In  order  to  use  the  survey  results  to  implement  improvements, 
it  will  be  necessary  to  further  develop  the  connection  between 
information,  analyses,  and  programmatic  action.  Additionally,  the 
Definition  of  Measurement  should  be  amended  to  remove  the 
statement  “and  hold  focus-group  meetings’*  unless  focus  groups  will 
actually  be  conducted. 
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5d.  Improvements  in  public  information  regarding  vehicle  delays  during  operations  as  well 
as  any  general  user  information  regarding  system  modifications,  route  changes,  and 
schedules. 


Goal: 

Improve  Passenner  Information. 

Pnrpnsn: 

Improve  all  aspects  of  public  information. 

Dofinitinii  of  Mnnsuromont; 

bnprovo  passenger  information  by  communication  of  service  problems  and  other  information  to  each 
voliicio,  the  station  platforms,  the  Telephone  information  Center,  media,  and  the  Service  Hotline 

Mothnd  of  Moosiironiont; 

Results  of  Rider  Surveys  will  indicate  success. 

Milostoiios: 

Rider  Survey  feedback.  (Wot  reported  in  FY2001  or  FY2002.) 

Fif)m  Muni  SpfWcn  Suiiulo(d%  Hofuut.  fY  2001  and  fY2002 


Reported  Performance  During  the  Audit  Period 

This  stniKl.ird  w;is  not  iticliidcd  in  the  Service  Standards  reports  in 
FY200I  or  FY2002. 

Verification  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

Muni  did  not  report  results  in  tliis  area  in  either  FY200I  or  FY2002. 
In  fact,  tlicre  are  no  specific  (|uestions  on  the  passenger  satisfaction 
survey  that  deal  wiili  customer  information.  Hov^ever,  during  the 
audit  period,  there  wore  a number  of  important  changes  to  the 
customer  information  and  pulilic  relations  functions  at  Muni  that 
support  this  goal: 

• Several  tliffereiu  functions  have  been  combined  under  an 
umlirella  group  responsible  for  Community  and  Public  Rela- 
tions. 

• This  department  has  developed  a “service  quality"  approach 
that  puts  Muni  staff  on  site  at  locations  whore  there  are  either 
significant  changes  in  service  or  unanticipated  service  delays 
or  disruptions  to  provide  information  and  assist  passengers. 


• The  Community  and  Public  Relations  department  is  integrated 
into  the  capital  projects  team  for  any  capital  project  that 
is  likely  to  cause  rider  and  neighborhood  disruptions.  The 
Community  and  Public  Relations  staffs  both  provide  advice 
to  the  project  delivery  team  and  take  lead  responsibility  for 
notifications  to  the  riders  and  the  larger  public  that  may  be 
affected  by  a Muni  project. 

• Muni  substantially  upgraded  its  website  and  provides  high  qual- 
ity information  regarding  service  changes,  schedule  updates 
and  disruptions. 

• During  the  audit  period.  Muni  printed  and  distributed  schedule 
books  for  all  lines  operated  by  the  railway  in  paper  format. as 
well  as  providing  telephone  and  electronic  information. 

• The  Next  Bus  system  was  installed  in  the  subway,  providing 
real  time  information  for  subway  riders.  In  addition,  a pilot 
Next  Bus  program  was  implemented  on  one  bus  route,  and  is 
planned  to  be  expanded  to  the  rest  of  the  bus  system,  pend- 
ing funding. 

• Automatic  stop  announcements  on  new  vehicles  provide  use- 
ful, new  information  to  passengers  on  board  Muni  vehicles. 
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While  Muni  has  clearly  made  important  strides  in  the  area  of 
community  and  public  relations,  it  has  yet  to  adopt  a definition  and 
method  of  measurement  that  can  adequately  describe  these  activities. 
Staff  proposes  the  following  changes  for  the  2003-04  fiscal  year. 

Verification  of  Reported  Performance  During  the 
Audit  Period 

Reported  performance  could  not  be  verified  since  this  standard  was 
not  included  in  performance  reports  in  FY200I  or  FY2002. 

Problems  or  Issues  Identified  During  the  Audit  Period 

One  of  the  reasons  this  service  standard  has  not  been  reported  on 
to  date  may  be  the  complexity  of  the  measure  itself.  The  measure 
"improvements  in  public  information  regarding  vehicle  delays  during 
operations  as  well  as  general  user  information  regarding  system 
modifications  route  changes  and  schedules"  combines  two  very 
different  types  of  information  - real  time  information  that  is  necessary 
for  customers  to  understand  and  respond  to  immediate  changes  and 
disruptions  in  the  system  and  information  with  a longer  lead  time, 
for  construction  projects,  schedule  and  service  changes,  etc.  These 
two  distinct  types  of  information  are  generally  not  handled  by  the 
same  people  or  the  same  departments,  which  has  made  measuring 
this  information  difficult. 

Muni  has  made  significant  progress  towards  improving  customer 
information,  both  for  real  time  and  longer  lead-time  information 
needs.  Significant  efforts  have  been  made  to  develop  a clear  chain 
of  command  for  handling  information  at  the  scene  of  a major  delay 
or  service  disruption.  Muni’s  intent  to  develop  a public  information 
3lan  that  covers  all  of  these  potential  information  needs  should  be 
encouraged.  This  plan  could  be  written  in  the  format  of  a "disaster 
ecovery  plan"  — identifying  all  of  the  steps  required  to  provide 
nformation  under  a variety  of  circumstances  and  who  is  responsible 


for  m.inagmg  th.n  .ispect  of  public  rel.itlons.  In  .1  well-developed 
f^an,  some  responsibilities  will  lie  with  operations,  centr.il  control, 
the  phone  room,  ,ind  community  rel.itions  service  te.tms.  Other 
responsibilities  will  belong  to  project  teams  working  on  route 
changes  and  capital  projects. 

While  Muni  has  made  substantial  progress  In  Improving  customer 
information,  it  has  not  developed  methods  for  evaluating  the  impact 
of  Its  Information  on  customers.  For  example,  when  there  Is  a major 
service  disruption  that  requires  Service  Quality  team  involvement, 
there  is  no  follow-up  with  passengers  to  sec  If  tliey  were  satisfied 
by  the  information  they  received.  Similarly,  In  neighborhoods  where 
construction  occurs,  Muni  has  worked  diligently  to  Improve  outreach, 
but  there  Is  no  feedback  loop  to  determine  whether  Information  was 
received  in  a timely  manner,  and  whether  It  was  hcl|)ful  to  riders. 
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Adopted  FY2004  Revision 


GonI: 

Improve  Passenger  Information. 

Piirposo; 

Improve  passenger  information  by  communication  of  service  problems  and  other  information  to  each  vehicle, 
the  station  platforms,  the  Telephone  Information  Center,  media  and  the  Service  Hotline,  and  assess. 

DfifiiHtinri  of  Monsiirumorit: 

Assess  current  practices,  develop  and  implement  improvement  plan. 

Mntlioil  of  Moosiirmnont; 

Plan  completed  and  implemented. 

Milostniios; 

Plan  completed  and  implemented. 

Ftom  Muni  Srirviro  Sunuianit  fiefwit.  I\t  Quoitrr  FY?004 


Action  Plan 

To  1)0  truly  usoful,  this  stnnclard  slioulcl  be  divided  into  two  goals 
to  distinguish  l)ctwoon  routine  information  provided  in  real  time 
and  longer-term  information  firovided  as  part  of  service  or  schedule 
cliango  and/or  construction  projects. 

Tiic  sto[)s  taken  to  modify  this  goal  in  2004  arc  a good  first  step 
towards  refining  this  measure.  Muni  is  encouraged  to  complete  its 
information  plan,  [)roviding  clear  direction  as  to  wliat  information 
needs  to  ho  provided  and  who  is  responsible  under  a variety  of 
circumstances.  1 his  includes  Identifying  a load  person  for  information 
provided  on  site,  as  well  as  leads  for  individual  tasks.  These  people 
should  not  l>e  named,  as  individuals  change  from  time  to  time,  but  by 
position  or  department.  Consideration  should  be  given  to  providing 
infoi  mation  in  accessilile  media  and  alternative  language  formats,  as 
Muni  serves  a diverse  customer  base. 

Muni  is  further  encouraged  to  develop  a method  of  assessing  the 
impact  of  their  information  on  the  riding  public  and  the  neighborhood 
or  community  impacteil  by  their  information  campaigns.  This  could 
come  in  tlie  form  of  a simple  survey,  implemented  by  Service  Quality 
teams  following  incidents  or  events  whore  public  information  has 
been  providoil. 
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To  further  differentiate  between  immediate  information  provided 
to  passengers  on  the  system  during  delays  or  other  unusual  events, 
and  longer-term  information  provided  to  address  construction  or 
other  longer-term  events  the  following  systems  of  measurement  are 
recommended: 


Goal: 

Improve  Real-Time  Passenger  Informalion  renardmn  service  delays. 

Purpose: 

rrovioe  the  public  with  information  regarding  service  delays  that  will  enable  them  to  make  choices  nhoiii  ihnii  nip 

Definition  of  Measurement: 

Iriiprove  passenger  information  by  communication  of  service  problems  and  other  iiiloimatiuii  to  each  vehicle,  the  station 
platforms,  the  Telephone  information  Center,  media,  the  Service  Hotline,  anil  Service  Quality  Teems.Operntors  will  make 
passenger  announcements  in  all  cases  where  service  delays  exceed  3 miiiutos  in  the  subway  and  111  mimites  on  siiifaco  lines. 
Central  Control  will  make  both  visual  and  verbal  announcements  in  all  stations  where  service  deleys  exceed  3 minutes  nn  ell 
LRV  lines.Service  Quality  Teams  will  be  assigned  to  all  unscheduled  sotvico  disruptions  oxcoodimi  two  hours 

Method  of  Measurement: 

Central  Control  logs. Passenger  Satisfaction  Surveys. 

Milestones: 

Plan  completed  and  implemented. 
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6d.  Efforts  to  improve  driver  training,  technical  as  well  as  accident  follow-up. 


Goal: 

100,000  hours  of  Driver  Training  per  year  and  5%  reduction  in  accidents. 

Purpose; 

Reduce  accidents  through  effective  operator  training  programs  as  well  as  effective  accident  follow-up  training. 

Definition  of  Monsiirement; 

Monthly  measurement  of  the  number  of  training  hours  by  type  of  class.  Track  reduction  in  accidents  as  a result 
of  more  effective  operator  training  and  accident  retraining.Training  hours  will  be  tracked  for  the  following  areas: 

• Now  Operator  Training 

• Immediate  Follow-up  Rides 

• One  Day  Accident  Retraining 

• Two  Day  Accident  Retraining 

• Verification  of  Transit  Training 

• Operator  Refresher 

• Passenger  Rolations/Conflict  Training 

Motlioil  of  MonsiirmnenC 

Ndinhor  of  FTA  roportahle  accidents  and  training  hours.  Data  will  be  reported  to  the  board  on  a quarterly  basis 

Milostonos: 

(JO, 000  hours  of  Driver  Training  and  G%  accident  reduction. 

Pi om  Muni  Sorvlco  Suiiulunh  Rejwn.  FY  2001  and  FY2002 


Reported  Performance 


There  were  of  1 40,692  and  1 29,769  operator  training  hours  in  FY200 1 and  FY2002,  respectively,  exceeding  the  goal  of  1 00,000  annual 

summarizes'this  information.  As  sho:. 

rJt  It^pi!;‘m  '''  based  on  FY2003 


Table  D-3  Operator  Training  Hours,  FY200 1 - FY2004 


FY2001 


^V:,. 


Total  Ojiorntor  Traiiiiiiji  Hours 


FY2002 


140,692 


Goal 


100,000 


FY2003 


Actual 


129,769 


1st  Quarter  2004 


Goal 

Actual 

Goal 

Actual 

FYTD  Goal 

100,000 

82,099 

50,000 

24,252 

50,000 

„ r-v-.-v ,iw<.rr/WJ  UMJ  IK  VlMrtfM 

Data  In  italics  have  not  horn  Aiiilltcil 

Table  D-4  Number  of  Accidents,  FY200 1 - FY2004 


FY2001 


Niimbnr  of  Accirinnts 


3,043 


FY2002 


2,880 


FY2003 


2,913 


Goal 


2,891 


Actual 


R'om  Muni  Savkc  StwM  KrfknUlh  Qinuicf  F)200l.  m002.  mm  und  /ji  0,«ri«  m004 
Data  in  Italics  have  not  boon  audited, 


2,966 


1st  Quarter  2004 


Goal 


2,767 


Actual 


7// 


FYTD  Goal 


704 
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Table  D-5  Accidents  per  I 00,000  Operator  Pay  Hours  Worked.  FY200 1 - FY2003 


FY2001  FY2002  FY2003 


Operator  Pay 
Mrs  Worked 

ft  of 

accidents 

Rate 

Operator  Pay 
Mrs  Worked 

^of 

accidents 

Rate 

channo 

Operator  Poy 
Mrs  Worked 

flccidonts 

RaU) 

% chmi()0 

Cable  car 

271,376 

116 

42.7 

292,654 

92 

31.4 

26.5% 

/-?5,455 

89 

70,9 

125.8% 

Green 

477,221 

424 

88.8 

478,227 

340 

71.1 

19.9% 

5B3.0W 

312 

53,5 

■2-18-':, 

Flynn 

388,118 

458 

118.0 

410.038 

417 

101.7 

13.8% 

369,460 

389 

105,3 

3.5% 

Kirkland 

502,677 

422 

83.9 

583,531 

439 

75.2 

10.4% 

499.378 

407 

81.5 

8,4% 

Potrero 

562,122 

655 

116.5 

614,385 

647 

105.3 

9.6% 

588,623 

682 

115,9 

10.1% 

Presidio 

470,249 

401 

85.2 

504,511 

435 

86.2 

1.2^ 

483,517 

455 

94.1 

9.2% 

Woods 

705,383 

567 

80.3 

774.054 

543 

70.2 

12.6% 

675,629 

632 

93,5 

33,2% 

Systemwide 

3,377,146 

3.043 

90.1 

3,657,400 

2,913 

79.6 

11.7% 

3,325,11? 

2.966 

89.2 

12. 1% 

From  Munj  Seoxe  Standardi  Rtport.40i  Quontt  FY200t.FY2002.fY2003 
Data  In  Italics  have  not  been  audited. 


In  addition  to  the  number  of  operator  training  hours  and  the  total 
number  of  accidents,  Muni  also  reported  the  number  of  accidents 
per  100,000  operator  pay  hours  worked  for  each  division.  In 
FY200l.the  system-wide  accident  rate  per  100,000  paid  operator 
hours  worked  decreased  by  4.8%  from  94.7  in  FY2000,  to  90.1  in 
FY200I.  Between  FY200I  and  FY2002,  the  system-wide  accident 
rate  decreased  by  I 1.7%  from  90.1  to  79.6.  However,  according  to 
unaudited  FY2003  data,  the  systemwide  accident  rate  increased  by 
12.1%  between  FY2002  and  FY2003.  Table  D-5  summarizes  this 
information 

Venficotfon  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

The  method  of  measurement  states  that  only  FTA  reportable 
accidents  should  be  tabulated.  In  fact,  all  accidents  were  counted 
in  fiscal  years  2001  and  2002.  (FTA  reportable  accidents  are  those 
accidents  that  result  in  a significant  level  of  property  damage,  injuries 
requiring  medical  attenuon  away  from  the  scene,  or  fatalities.) 


All  other  information  was  collected  according  to  the  dcscrihod 
definition  and  method  of  measurement.  The  number  of  training  hours 
was  calculated  based  on  the  number  of  Individuals  who  coinploied 
a particular  training  session  multiplied  by  the  length  (In  liours)  of 
the  training  session.  One-  and  two-day  accident  rotialning  was 
not  reported  separately;  Instead,  the  number  of  hours  for  one-  and 
two-day  training  were  combined  and  the  toi.il  number  of  accident 
retraining  hours  was  reported. 

As  of  July  I.  2002,  all  accident  data  was  tracked  using  the  Tiansit 
Safe  Daubase.  Accident  information  was  entered  directly  into  ilic 
database  by  transit  operators  and  investigating  supervisors.  Prior 
to  this  time,  accident  reports  were  filed  In  hard  cofjy  and  were  then 
entered  into  the  old  BMIS  mainframe  daubase. 
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Verification  of  Reported  Performance  During  the 
Audit  Period 

An  audit  of  operator  training  data  for  the  first  quarter  of  FY2002 
showed  that  data  were  reported  correctly  for  this  period,  with  the 
exception  of  Violence  in  the  Workplace  training.  Violence  in  the 
Work[)lace  training  hours  were  double  counted  since  they  were 
counted  as  part  of  New  Operator  Training  and  were  then  counted 
again  separately.  Thus,  the  total  number  of  operator  training  hours 
for  FY200I  should  be  I38,6')2,nnd  should  be  129,038  for  FY2002. 

Data  for  the  number  of  accidents  and  the  accident  rate  per  100,000 
o()crator  pay  hours  worked  were  verified  for  fiscal  years  2000, 2001 
and  2002.  Source  data  for  operator  training  hours,  accidents,  and 
o[)crator  pay  hours  worked  consisted  of  summary  spreadsheets 
that  had  been  compiled  by  staff  at  the  time  the  Sei  vice  Standaids 
Rc[)orts  were  bcin^;  written. 

Problems  or  Issues  Identified  During  the  Audit  Period 

All  iiccldonts  wore  roportod  In  FY200I  nnd  FY2002,  not  just  those 
that  were  reportable  under  FTA  guidelines.  Muni  lias  already 
cliangcd  the  nictliod  of  tncasureniont  for  this  indicator  to  include  all 
accidents,  not  just  those  that  arc  FTA  reportable. 

Violence  in  the  Workplace  training  hours  were  double  counted  in 
botli  fiscal  years  2001  and  2002.  Violence  in  the  Workplace  training 
hours  were  counted  as  part  of  New  Operatoi'  Training  and  were 
then  counted  again  separately.  Thus,  the  total  number  of  operator 
training  houi  s for  FY200I  should  be  I38.6‘f2  and  should  be  129.038 
for  FY2002. 

The  reporting  of  accident  rates  as  calculated  per  100.000  operator 
pay  l)ours  worked  is  not  a standard  measurement  in  the  industry. 
The  transit  industry  standard  is  to  report  accident  rates  per  miles 
driven,  hours  of  service  provided,  or  period  of  time.  Furtfiermore, 
according  to  the  MOU  between  Muni  and  the  Transit  Workers 


Union  operator  incentives  are  dependent  on  the  reduction  in  the 
total  number  of  accidents  for  all  categories  of  revenue  accidents," 
not  a reduction  in  the  accident  rate  per  1 00,000  operator  pay  hours 
worked.  Thus,  there  is  no  clear  reason  as  to  why  the  accident  rate 
should  be  reported  in  a manner  that  is  inconsistent  with  industry 

Standards. 

In  both  FY200I  and  FY2002.  Muni  failed  to  achieve  its  goal  of  a 
5%  reduction  in  the  number  of  accidents.  This  trend  appears  to  be 
continuing  in  FY2003. 

Changes  in  Measurement  Since  the  Audit  Period 

Subsequent  to  the  audit  period.  Muni  has  adjusted  the  milestone  for 
training  and  accidents  as  follows: 

• FY2003  and  FY2004  - 50,000  hours/5%  Accident  Reduction 

• FY2004  Method  of  Measurement  was  amended  to:“Number  of 
reportable  accidents  and  training  hours.  Data  will  be  reported 
to  the  board  on  a quarterly  basis.” 

Act/on  Plan 

The  following  are  relatively  minor  issues  that  should  be  addressed: 

• Violence  in  the  Workplace  training  hours  should  not  be  double 
counted  in  reports  of  operator  training  hours. 

• Ensure  that  the  definition  of  measurement  accurately  describes 
the  types  of  accidents  being  reported. 

“ Consider  reporting  the  accident  rate  in  a manner  consistent 
with  transit  industry  standards,  such  as  accidents  per  miles 
driven,  hours  or  service  provided  or  over  a given  period  of 
time.  Differentiate  between  vehicle  accidents  (collisions)  and 
customer  accidents  (e.g.,  slips  and  falls).  Table  D-6  shows  the 
systemwide  accident  rate  per  1 00.000  revenue  miles  for  fiscal 
years  2001, 2002  and  2003. 
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Table  D-6  Accidents  per  100,000  Revenue  Miles. 
FY200I  - FY2003 


Systemwide  Performance 

FY2001 

FY2002 

FY2003 

Revenue  Miles 

24,735,581 

26,613,598 

28.767.205 

Number  of  Accidents 

3,043 

2,913 

2.966 

Accident  Rate  per  100,000 
Revenue  Miles 

12.30 

10.95 

10.31 

From  Mu/>»  Service  Standards  Repon,4th  Quantr  FY200I,  fY2002,  FY2003  and  1st  Qoott$r  FY2004 
Data  n ttaUs  hare  not  been  audited 


Achieving  Goals  for  Accident  Reduction 

According  co  the  Office  of  Health  and  Safety  Manager,  hours  of  driver 
training  are  not  having  a direct  impact  on  accident  numbers  and  rates. 
In  fact,  Muni  has  an  excellent  driver  training  program  that  conforms 
to  all  national  and  state  guidelines  and  standards.  Rather  than  focus 
on  driver  training,  a comprehensive  approach  and  coordinated  effort 
to  reduce  accidents  across  divisions  and  departments  is  needed.  In 
order  to  reduce  the  number  of  accidents,  safety  needs  to  be  made 
the  top  priority  throughout  the  system,  above  all  other  goals  and 
objectives.  A solid  culture  of  safety  needs  to  be  established  before 
other  performance  goals  can  be  accomplished,  otherwise  Muni  will 
continue  to  be  unable  to  achieve  its  goals  with  regard  to  accident 
reduction. 

There  are  three  key  elements  to  reducing  accidents: 

• Engineering  improvements  (i.e.,  remove  safety  problems 
through  engineering  and  technical  improvements); 
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Employing  warning  devices  and  instituting  safety  rules  and 
procedures,  including  safety  training;  and 

• Enforcing  safely  rules  and  procedures. 

Interviews  with  staff  indicate  that  Mum  Is  doing  well  with  regard  to 
the  first  two  elements, but  has  failed  with  regard  to  strict  enforcement 
of  safety  rules  and  procedures  and  In  creating  an  organizational 
culture  that  makes  safety  the  top  prior  ity.  Work  has  alr  eady  begun 
on  addressing  these  Issues,  and  the  auditor  recommends  that  this 
continue  to  be  a focus  for  both  management  and  transit  operators. 

The  number  of  accidents  and  .iccideiu  r ate  is  closely  related  to 
problems  with  on-irmc  performance  and  headway  adhci*ence. 
The  fact  that  a significant  number  of  lines  are  r unning  cai  ly/shai  p 
may  mean  that  drivers  arc  speeding,  or  driving  In  other  ways  that 
compromise  safety.  More  str  eet  Inspectors  arc  needed  to  enforce 
on-time  performance.  Run  times  and  schedules  may  need  to  be 
reevaluated  to  ensure  that  drivers  can  complete  their  r outes  both 
safely  and  on  time. 


Proposition  E Municipal  Transportation  Quality  Review 

16.  Number  of  crime  incidents  on 


DRAFT  Final  Report 


Municipal  Railway  vehicles  or  In  Municipal  Railway 


fuemuK:^, 

Goal: 

Piirposo: 

Dofinition  of  Monsiiromont: 

Roduction  ol  G%  from  previous  year.  

fo  moasuro  the  crime  rate  on  transit  vehicles. 

Quorlorly,  wo  report  on  nil  cateporios  of  crime  incidents  witn  the  corresponding  quarter  for  the 

iirovinus  year  with  a % change.  — — ^ — 

Motliod  of  Monsiiromont: 

tlriia  is  collocled  hy  the  Muni  Transit  Police.  Data  will  be  reported  to  the  board  on  a quarterly  basis._ 

Milostoiios:  F)%  

fiotu  Muni  Stuvifo  StotuloKlt  Krf/oit,  FY  2001  nnd  fYlOO? 


Reported  Performance 

The  luimhcr  of  repor  ted  crinio  incidents  includes  nil  incidents  that 
occurred  on  Muni  vehicles  or  property.  Tnl)lc  D-7  sutninarizcs 
rc[)ortcd  [)erforniiince  in  FY200I,  FY2002,  FY2003  and  the  first 
quarter  of  FY200^  with  regard  to  crime  incidents.  While  Muni 
failed  to  meet  its  goal  In  FY200 1 . It  exceeded  its  FY2002  goal  with  a 
reduction  of  an  additional  126  incidents.  The  FY2003  goal  was  also 
exceeded  hy  the  reduction  of  an  additional  121  incidents. 

Table  D-7  Number  of  Reported  Crime  Incidents,  FY200  I - FY2004 


Ftx^tu  Muftl  Stiim/jifi/s  FYJOOl,  FY2002,  FY2003  ond  lit  QiMitft  rY2004 

DatA  In  Italics  have  not  I’con  aiulltci). 
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Verification  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

Incident  data  was  tracked  by  both  Muni’s  Security  Department  and 
by  the  San  Francisco  Police  Department  (SFPD).  Incident  reports 
were  generated  by  Muni  Central  Control,  Passenger  Services  and  the 
SFPD.  Those  generated  by  Central  Control  and  Passenger  Services 
were  logged  into  the  Transit  Safe  database  by  Muni  staff. 

The  SFPD  not  only  tracks  police  reports,  but  also  receives  copies  of 
incident  reports  from  Muni.  The  SFPD  maintained  its  own  incident 
database,  and  provided  copies  on  a regular  basis  to  Muni  staff  so  that 
SFPD  information  could  be  entered  into  Transit  Safe.  Muni  staff  also 
double  checked  their  incident  records  against  those  of  the  SFPD  to 
make  sure  all  information  was  consistent,  (Beginning  in  2004,  the 
SFPD  will  enter  incident  information  directly  into  the  Transit  Safe 
database.) 

Verification  of  Reported  Performance  During  the 
Audit  Period 

An  audit  of  data  for  FY2002  found  that  data  appeared  to  be  correctly 
reported  for  the  first,  third  and  fourth  quarters  of  the  fiscal  year. 
However,  there  were  slight  data  discrepancies  for  the  second  quarter 
Staff  stated  that  this  discrepancy  was  likely  due  to  the  fact  that  some 
additional  incident  reports  came  in  and  were  added  to  their  tracking 
spreadsheet  after  the  incident  data  was  submitted  for  publishing  in 
the  Service  Standards  Report. 


2nd  Ouartnr  FY02 


Reported 

Actual 

Number  of  Incidents 

676 

688 
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More  significant  is  the  fact  that  fare  citations  issued  by  Muni  fare 
inspectors  were  not  included  in  the  FY200I  and  FY2002  incident 
reports.  According  to  the  Security  Department,  only  fare  citations 
issued  by  the  SFPD  were  counted.  The  following  table  shows  tlio 
number  of  fare  citations  issued  by  Muni  fare  inspectors  in  FY2001 
and  FY2002.  A total  of  4,146  and  3,660  fare  citations  were  given  in 
FY2001  and  FY2002,  respectively.  In  comparison,  only  66  and  1 6 fare 
evasion  incidents  were  reported  in  the  FY200I  and  FY2002  Service 
Standards  Reports,  respectively. 

Table  D-8  Number  of  Fare  Citations  Issued  by  Muni  Fare 
Inspectors 


FY7001 

FY20U2 

1st  Quarter 

926 

726 

2nd  Quarter 

828 

713 

3rd  Quarter 

1,522 

803 

4th  Quarter 

870 

1,358 

TOTAL 

4.140 

3.GG0 

Problems  or  Issues  Identified  During  t/»c  Audit  Period 

Fare  evasion  citations  Issued  by  Muni  fare  Inspectors  were  not 
included  in  the  FY200I  and  FY2002  Service  Standards  Reports. 
Including  this  data  increases  the  number  of  reported  Incidents 
significantly,  but  does  not  affect  the  achievement  of  a 5%  reduction 
in  incidents  for  FY2002.  (With  the  addition  of  fare  citations,  the  total 
number  of  incidents  in  FY200I  was  7,073  and  was  6,315  In  FY2002. 
This  represents  a 10.7%  reduction  in  incidents  from  FY200I  to 
FY2002.) 

The  SFPD's  maintenance  of  a separate  Incident  daubasc  has  made  it 
more  time  consuming  and  labor  intensive  for  Muni  staff  to  generate 
incident  reports.  This  issue  should  be  resolved  in  2004  when  the 
SFPD  will  begin  entering  information  directly  into  the  Transit  Safe 
database  as  well. 
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While  Transit  Safe  allows  staff  to  track  details  about  ^'1 

cannot  automatically  generate  some  of  3;^; 

example,  total  incidents  can  only  be  tabulated  for  each  ^''^^lon. rather 

than  for  the  system  as  a whole.  Additionally.  ^ 

generate  reports  about  the  total  number  of 

tabulate  this  information,  staff  must  look  up  each  individual  case  to 
determine  whether  or  not  it  was  an  assault  or  arrest. 

Coding  of  incidents  is  not  standardized  across  all  Muni  departments. 
This  sometimes  leads  to  Inadetpiate  or  inaccurate  information  on 
incident  reports. 

Changes  In  Measurement  Since  the  Audit  Period 

Subse(|uent  to  the  audit  period.  Muni  has  made  no  changes  to 
the  definition  of  measurement,  method  of  meastirement,  nor  the 
milestones. 

Auditor  Recommendations 

Fare  evasion  citations  issued  by  Muni  fare  inspectors  should  be 
included  in  the  incident  reports  included  in  the  Service  Standard 
Reports. 

A consistent  method  of  coding  crime  incidents  sliould  ho  developed 
and  used  hy  all  departments  rcsponsihle  for  collecting  this  information. 
Additionally,  new  t|uerics  should  ho  developed  within  Transit  Safe  to 
streamline  and  automate  the  reporting  of  crime  incident  data. 

The  indicator  purpose  inclutled  in  the  Service  Standard  Report 
should  he  amended  to.'To  measure  the  crime  rate  on  transit  vehicles 
and  facilities."  This  more  accurately  reflects  both  the  content  and 
purpose  of  the  iiulicator. 
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£.  Employee  Satisfaction 

Employee  satisfaction  not  only  affects  staffing  performance,  but  also 
affects  customer  service  and  overall  system  performance  as  well, 
since  dissatisfied  employees  are  less  likely  to  provide  the  best  service 
possible.  Consequently.  Proposition  E included  the  following  service 
standards  aimed  at  measuring  employee  satisfaction; 

• le.  Number  of  grievances. 

• 2e.  Speed  of  resolution  of  grievances. 

• 3e.  Longevity  of  employment. 

• 4e.  Employee  recognition. 

• 5e.  Employee  education  and  training  opportunities. 

Key  Recommendations 

The  following  are  key  recommendations  based  on  the  findings  of  our 
audit  of  FY200I  and  FY2002  Service  Standards  data  and  on  trends 
observed  in  unaudited  FY2003  and  FY2004  Service  Standards  data: 

• In  addition  to  the  service  standards  listed  above,  Muni  is  also 
measuring  employee  satisfaction  via  its  annual  Employee  Sur- 
vey. The  results  of  this  survey  should  be  incorporated  into  this 
group  of  service  standards  since,  in  many  respects,  it  provides 
a better  measure  of  employee  satisfaction  than  the  current 
group  of  service  standards. 

• For  Service  Standard  I e,  a goal  or  benchmark  for  the  number 
and  type  of  grievances  should  be  established  so  that  man- 
agement can  more  effectively  monitor  changes  in  grievance 
rates. 
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Foi  Service  Standard  2e,  the  method  for  reporting  speed  of 
I esolution  of  grievances  for  non-opei'ator  grievances  needs  to 
be  closely  examined  and  monitored  to  ensui  e that  informa- 
tion is  reported  correctly  in  the  future. 

• Service  Standard  3e  provides  almost  no  information  about 
employee  satisfaction.  If  anything,  It  is  more  a measure  of 
staffing  performance,  since  long-term  employees  (and  relatively 
minimal  staff  turnover)  reduce  the  need  to  hire  and  train 
new  employees.  The  auditor  therefore  recommends  that  this 
measure  be  eliminated  from  the  Service  Standards  Report,  or, 
if  retained,  moved  to  the  Staffing  Performance  section. 

• For  Service  Standard  4e,  the  actual  number  and  typo  of  annual 
and  quarterly  awards  made  should  be  Included  in  the  Service 
Standards  Report. 

• For  Service  Standard  5c,  all  employee  training  hours  should 
be  tracked  in  some  type  of  centrallxetl  system  in  order  to 
facilitate  future  reports  and  audits. 
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/e. 


/vumot:#  u/ 

Goal: 

Piirposo: 

Dofinition  of  Monsiiromonf: 

ReDorl  tiiiarlerly  on  the  number  of  grievances. 

jjiidriuiY  IMI_^ ^ T.r.nrmiiHrHntnfnr  the  hoard  nn  a Quarterly  basis. 

Motlioil  n(  Monsiirmnont: 
Milostonos: 

Internal  trackimi  system  will  be  used  to  provide  d^ 

Quarterly  (jriovance  report 

Prom  Mi/nl  Scfvifp  Stofifioffi*  fY  7001  and  fY7007 


Reported  Performance  emoloyee  complaints.  In  FY200I,  and  the 

This  s.»,i,l...sl  n.c..s,i,  cs  j.  losahcos  hlo.l  b?  M.i.ii  maintenahce  and  other  employee  grievances  were 

first  .inartor  ol  FY2002,  only  transit  operatoi  giiovances  „ievances  were  filed  in  FY2003.  Tables  E- 1 and  E-2summariie 



^F?™  I . then  was  a backlog  ol  over  300  grievances  that  were  resolved  over  the  conrse  of  the  fiscal  year. 


Table  E-l  Operator  Grievances,  FY200 1 - FY2004 


Griovmico  Status 

FY2nin  Totnl 

FY2002  Totnl 

FY2003  Totnl 

1st  Q 2004 

Rocnivud 

93 

117 

60 

7 

i^nsolvod 

4119 

124 

62 

7 

Aclivn 

9 

2 

0 

0 

Prom  AiunI  Smm/mJi  Quam  FY700IJY7002.fY7003  and  lit  Qimt.r  FY2004 

D.ua  in  lullcs  h.wt>  not  Inmn  amlltoil 

Table  E-2  Maintenance  and  Miscellaneous  Employee  Grievances,  FY200I  - FY2004 


Griavnnoc  Status 

FY2002  Totnl 

FY2003  Totnl 

1st Q 2004 

Rocoivnd 

17 

14 

1 

Rosolvod 

13 

16 

1 

Active 

4 

2 

2 (from  prev.  FY) 

Fi-om  Mml  Scivico  StoMs  Qih»t«  fViOOl,  mOOi.  mm  .ind  /«  QinuM  mm 


Data  In  Italics  have  not  bot'n  aiKlitcd, 
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Verification  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

The  Human  Resources  Deparimcnt  tracked  grievances  on  a 
quarterly  basis.  All  data  was  collected  and  reported  according  to 
the  definition  and  method  of  measurement.  The  only  exception  was 
that  grievances  were  not  reported  for  each  division;  instead,  the  total 
number  of  operator  and  non-operator  grievances  were  reported. 

Verification  of  Reported  Performance  During  the 
Audit  Period 

Based  on  an  audit  of  fourth  quarter  FY2002  data  for  the  number  of 
maintenance  and  miscellaneous  employee  grievances,  and  FY2002 
quarterly  data  for  operator  grievances,  data  appear  to  have  been 
collected  and  reported  accurately. 

FY200I  operator  grievance  data  could  not  be  verified  due  to  the 
1 fact  that  the  original  grievance  reports  no  longer  exist.  The  auditor 
I did  verify  that  the  backlog  of  grievances  had  been  resolved  by  the 
end  of  FY200I. 

Problems  or  Issues  Identified  During  the  Audit  Period 

. Grievances  were  not  reported  by  division,  as  described  in  the 
definition  of  measurement. 

Changes  in  Measurement  Since  the  Audit  Period 

Subsequent  to  the  audit  period,  Muni  has  made  no  changes  in 
the  definition  of  measurement,  method  of  measurement,  nor 
milestones. 
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Auditor  Recommendations 

Either  the  definition  of  mcasuromoni  should  bo  amended  so  that  it 
no  longer  states  that  grievances  will  be  tracked  by  division,  or  the 
number  of  grievances  for  each  division  shouid  be  reported  separately. 
Additionaliy.no  goai  has  been  established  for  the  number  or  type  of 
grievances  that  are  filed,  and  it  is  thus  difficult  to  use  tlds  measure 
as  an  indication  of  employee  satisfaction.  The  auditor  recommends 
that  a goal  or  benchmark  for  the  number  and  type  of  grievances 
bo  established  so  that  management  can  more  effectively  monitor 
changes  in  grievance  rates. 
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2e. 


Speed  of  resolution  of  grievances. 


Goal: 

Rnsolvo  7G%  o(  internal  ijrievances  within  30  days.  

Piirposo: 

Moasurij  llio  effoctiveness  of  the  Labor  Relations  in  the  resolution  ul  yuKva..^ 

Dof iiiif imi  nf  Mnn5viirf!niont^ 

Montlilv  measurement  of  the  resolution  of  grievances.  — 

Motlioil  of  Monsiirunumt: 

InlBinal  trackiiKi  system  will  be  used  to  provide  data  for  the  board  on  a quarterly  basjs^ 

Milnstnnos; 

Rnsniiition  of  75%  nf  grievances  within  30  days.  1 

Trofn  Muni  Service  Sttim/onh  fleiiort.  FY  2001  or\<l  FY2007 


Reported  Performance 

This  staiul.inl  mensut  os  tho  speed  with  which  employee  grievances 
arc  resolved.  In  FY200I  and  the  first  (quarter  of  FY2002,  only 
transit  opcr.itor  grievances  were  tracked,  while  both  operator  and 
maintenance  and  other  employee  grievances  were  monitored  for 
tlic  last  three  (|iiartcrs  of  FY2002.  However,  due  to  tlie  backlog  of 
over  300  grievances  at  the  beginning  of  FY200l,speed  of  resolution 
of  grievances  were  not  recorded  for  the  first,  second  and  third 
(]uarters.  Alternatively,  the  Service  Standards  Reports  described  the 
steps  that  had  been  taken  to  speed  the  resolution  of  grievances 
during  the  first,  second  and  third  quarters  of  FY200I. 

For  all  quarters  in  wliich  data  were  reported,  75%  or  more  of 
operatoi' grievances  were  resolved  within  30  days.  For  maintenance/ 
miscellaneous  employees,  100%  of  grievances  were  resolved  within 
30  (lays  during  the  second  (|uartor  of  FY2002,  wliile  0?o  and  30% 
wore  resolved  within  30  days  for  the  thiid  and  fourth  quarters  of 
FY2002,  respectively.  In  the  fourth  quarter  of  FY2003,  97.5%  of 
ojicrator  grievances  were  resolved  within  30  days,  wldlo  only  33% 
of  maintcnancc/miscellaneous  employee  grievances  were  resolved 
within  30  days. 


Verification  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

The  timeline  for  Human  Resources’  grievance  tracking  begins  when 
an  operator  grievance  reaches  Step  3,  and  when  a maintenance/ 
miscellaneous  employee  grievance  reaches  Step  2.  Grievances  were 
assigned  a number,  and  major  actions  and  associated  dates  were 
logged  by  Human  Resources  staff. 

The  time  spent  to  resolve  operator  grievances  was  based  on  when 
a Step  3 decision  was  issued.  When  time  was  waived  by  mutual 
consent  of  either  party,  the“clock  was  stopped"  and  the  waived  time 
was  not  counted.  Non-operator  grievances  were  only  considerecJ 
to  be  resolved  if  they  were  resolved  at  the  Step  2 level.  These 
timelines  are  based  on  the  fact  that  grievances  that  progress  beyond 
the  Step  3 (for  operators)  or  Step  2 levels  (for  non-operators)  are 
no  longer  the  primary  responsibility  of  the  Muni  Human  Resources 
Department. 
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Verification  of  Reported  Performance  During  the 
Audit  Period 

An  audit  of  operator  grievance  data  for  FY2002  verified  that  all  data 
was  collected  and  reported  correctly.  Non-operator  data  for  FY02 
could  not  be  verified.  Human  Resources  staff  stated  that  they  did 
not  report  speed  of  resolution  information  for  FY2002  and  did  not 
know  how  the  data  were  generated. 

Problems  or  Issues  Identified  During  the  Audit  Period 

Data  on  the  speed  of  resolution  of  grievances  for  non-operators 
could  not  be  verified.  It  is  unclear  as  to  how  this  information  was 
measured. 

Changes  in  Measurement  Since  the  Audit  Period 

Subsecjuent  to  the  audit  period,  Muni  has  made  no  changes  in 
the  definition  of  measurement,  method  of  measurement,  nor 
milestones. 

Auditor  Recommendations 

The  method  for  reporting  speed  of  resolution  of  grievances  for  non- 
operator grievances  needs  to  be  closely  examined  and  monitored 
to  ensure  that  information  is  reported  correctly  in  the  future.  For 
FY2002.this  information  appears  to  have  been  reported  incorrectly. 
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3e.  Longevity  of  employment. 


Gonl‘ 

providG  an  annual  renort  of  the  lonnevity  of  employment. 

Piirnoso: 

Track  and  report  on  the  longevity  of  employment. 

Dofinition  of  Monsuromont: 

Monthly  tracking  of  the  number  of  employees  that  leave 
throiKih  retirements,  terminations  and  promotions. 

Molhoil  of  Monsuromont: 

Produce  an  annual  report  that  shows  the  longevity  ot 
nniiilovmont  by  classification,  including  Years  of  Service. 

Milostonos: 

Annual  longevity  of  omployment  report. 

f lOfii  Mtinl  Soivlctt  Stoiuliinli  Ht>poit,  fY  2001  ond  fY2002 


Reported  Performance  During  the  Audit  Period 

Tliorc-  was  a significant  jump  In  longevity  of  employment  for  nearly  all  job  categories  between  FY2001  and  FY2002.  This  was  due  to  the  fact 
dIro,ryears  of  service  llth  M.mi  were  cotntted  in  FY2001,  and  that  years  of  service  with  the  City  of  San  Francisco  were  counted  for 
FY2002  (an.l  all  sul.sequent  fiscal  years),  It  sitould  be  noted  that  certain  job  classes  that  are  associated  only  with  transit  operations  are  Mum- 
specific,  and  longevity  thus  reflects  only  years  of  service  with  Muni,  Table  E-3  summarizes  employee  longevity  for  these  two  years. 


Tabic  E-3  Employee  Longevity,  FY200  I - FY2002 


Jnh  Cntonmy 

FY2001 

FY2002 

Ollico/Cloiicnl 

I.H3  ynms 

1 1.52  years 

(Hficiiils  iiml  Administintms 

1.33  yoms 

14.01  yonrs 

Pmaprofossinniils 

0.73  ynnrs 

1.00  years 

Profossionnls 

2.00  yonrs 

6.50  years 

Piotoclivo  Sorvicn 

0.G8  years 

5.00  years 

Snivico  Mnintnnimco 

0.50  years 

10.64  years' 

Trnnsil  tlpariilms 

10.82  years 

- 

Skillmi  Crnlt 

0.02  years 

12.00  years 

Tocimiciims 

1.00  yoms 

7.03  years 

Fi'on^  Alunf  Sefvitc  QiMtttr  fY200l  and  fY2002 

*lncludos  hoih  sri  vice  n\Alf«oiunce  Ami  u Atwli  opeuiors  for  F'*  2002 
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Verification  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

The  Position  Control  System  (PCS)  database  was  used  to  track  all 
Muni  personnel  data,  including  changes  in  appointment,  temporary 
and  permanent  separations  (by  type),  etc.  The  PCS  database  was 
used  to  generate  a report  on  the  average  years  of  Muni  service  in 
each  job  class  and  category. 

Verification  of  Reported  Performance  During  the 
Audit  Period 

Based  on  an  audit  of  FY200I  and  FY2002  data,  all  data  on  employee 
longevity  appears  to  have  been  reported  accurately. 

Problems  or  Issues  Identified  During  the  Audit  Period 

Only  years  of  service  with  Muni  were  reported  in  FY200I,  while 
total  years  of  service  with  the  City  of  San  Francisco  were  reported 
for  all  non-Muni  specific  job  categories  in  FY2002. 

Changes  In  Measurement  Since  the  Audit  Period 

Subsequent  to  the  audit  period,  Muni  has  made  no  changes  in 
the  definition  of  measurement,  method  of  measurement,  nor 
milestones. 
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Auditor  Recommendations 

The  Service  Standards  Reports  show"Avcrage Years  of  Muni  Service." 
This  statement  is  incorrect,  since  the  longevity  report  actually  shows 
average  years  of  service  with  the  City  of  San  Francisco  for  all  job 
categories  that  arc  not  Muni-spccific  (e  g. .Transit  Operators).  Either 
this  label  should  be  changed  to  accurately  reflect  the  Information 
that  is  reported,  or  only  years  of  Muni  service  should  be  calculated 
and  reported. 

This  Service  Standard  provides  almost  no  Information  about 
employee  satisfaction.  If  anything,  it  Is  more  a measure  of  staffing 
performance,  since  long-term  employees  (and  relatively  minimal  staff 
turnover)  reduce  the  need  to  hire  and  train  new  employees.  The 
auditor  therefore  recommends  that  this  measure  be  eliminated  from 
the  Service  Standards  Report,  or.  if  retained,  moved  to  the  Staffing 
Performance  section. 
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Employee  recognition.  1 

Goal: 

' Annual  achiovement  of  honoroas  in  the  following  program., 

• (121  Syslemwiile  Operators  of  the  Month  Award 

• |4|  Transit  Supervisor's  of  the  Quarter  Award 

. (4)  Finance  & Admin  Employee's  of  the  Quarter 

• (12)  Maintenance  Employees  of  the  Month 

• (4)  Maintenance  Supervisors  of  the  Quarter 

• (4)  Safety  ft  Training  Employee  of  the  Quarter 

• W Accessihility  Employee  of  the  Ouarte; nerformance! 

Piirposn: 

nofinitiim  of  Monsiirnmoiil; 

To  rocoiinizo  iho  acliiGvcmenIs  of  employ ^ nomination  evaluation 

Moiilhly  trnckinii  of  all  award  programs.  Awar  progr  includes  but  is  not  limited  to  employee's 

.r,  H.  1..  Tto  d ..  EEO  measu,...  For  Operator 

P.rf.r,..r,o....l.ra.-, ;«M.rl.»a»oe^ 

Molliml  of  Moasuimnont: 
Milostnnns: 

Aniuinl  nchievonienl.  

Crom  Mim/  Rrporf.  ry  JQOI  tuul  fY2002 


Reported  Performance 

Tlicre  was  no  reported  performance  for  this  indicator  in  either  tlie 
FY200I  or  FY2002  Service  Standards  Reports. 

Venfient/on  of  Definition  and  Method  of 
Measurement  Durins;  the  Audit  Period 

Tlie  definition  and  method  of  measurement  could  not  be  verified. 
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Verification  of  Reported  Performance  During  the 
Audit  Period 

The  following  awards  were  made  during  calendar  year  2001: 

• (10)  Systemwide  Operators  of  the  Month  Award 

• (2)  Transit  Supervisor's  of  the  Quarter  Award 

• (3)  Finance  & Admin  Employee's  of  the  Quarter 

• (15)  Maintenance  Employees  of  the  Month 

• (0)  Maintenance  Supervisors  of  the  Quarter 

• (2)  Safety  & Training  Employee  of  the  Quarter 

• (0)  Accessibility  Employee  of  the  Quarter 

Additional  awards  were  made  for  Construction  Division  Employees 
of  the  Quarter  (2).  Capita!  Planning  and  Legislative  Affairs  Division 
Employee  of  the  Quarter  (3), and  Human  Resources.  Labor  Relations, 
and  EEO  Division  Employee  of  the  Quarter  (3). 

All  honorees  received  formal  recognition  at  an  awards  banquet, 
gift  certificates,  and  were  eligible  to  win  major  prizes.  All  those 
nominated  for  awards  were  recognized  as  well. 

Problems  or  Issues  Identified  During  the  Audit  Period 

The  Service  Standards  Reports  did  not  report  on  the  actual  number 
and  type  of  awards  made  during  each  fiscal  year,  although  some  of 
the  goals  were  met 

Chonges  in  Measurement  Since  the  Audit  Period 

Subsequent  to  the  audit  period,  Muni  has  made  no  changes  in 
the  definition  of  measurement,  method  of  measurement,  nor 
milestones. 
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Auditor  Recommendotions 

The  actual  number  and  type  of  annual  and  quarterly  awards  made 
should  be  included  in  the  Service  Standards  Report. 
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Employee  educot 

ion  ana  iraunn^  — — 



Goal: 

Piirnosa: 

rfOVlOO  JUjUU^  *>**"“°' * 

nnnoflunities  for  employee  development 

~T.. . ,1  fnr  tho  fnilowinQ  arG3S. 

Dofinilinn  of  Monsiirnmont; 

Tnifiiiwi  hours  wi  1 hc  tfsckcu  JllUllllny  10  . 1 • • 1 

^ I-I*. 

. /llahilsolHiol'ly  Effective  People 

• Ambassmior  Training 

• Siiporvisory  Skills  Training 

• Mnnagonient  Skills  Training  : 

. Vioicneo  in  Iho  Workplace 

• Dnsklop  Comiuiler  Training 

• AOdlllOnai  U<Mmil»  <»a  — miortorlu  basis 

: nata  will  be  reported  to  the  board  on  a quarterly  oasis. 

Mothod  ol  Mansmnmnnt: 
Milnstomis: 

[rack  nimibor  ol  hours  by  tvpe  ol  training.  uaid_vvm 

r)U,unO  annual  hours  training  anti  education — 

lom  Miiiil  Si'ivlir  Simitiii'l‘  M'""' 


Reported  Performance 


Tills  su,ul..rd  nuMsuros  tr. lining  lioui  s for 

FY200I  53.296  hours  of  training  were  provided,  and  50.880  houis 
of  training  wore  provUlod  In  FY2002.  In  FY2003,  3 1 ,24 1 hours  of 
training  wore  provkloil  (loss  than  the  goal  of  50,000  annual  training 
hours):  ,111(1  7,169  hours  wore  offered  In  the  first  ciuartcr  of  FY2004. 
Training  was  provided  in  all  of  the  categories  outlined  in  the 
definititan  of  iiioasuroniont  with  the  exception  of  the  "7  Habits  of 
I lighly  Effective  People"  training  which  was  not  provided  in  FY2002 

ami  FY2003, 


Verification  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

Maintenance  and  administrative  training  were  tracked  in  a number 
of  ways.  Administrative  training  includes  regular  in-house  training 
sessions  as  well  as  conferences,  retreats,  and  seminars  attended  by 
Staff  members. 

For  in-house  training,  the  number  of  people  who  completed  each 
class  was  recorded  and  the  total  number  of  training  hours  w 
computed  by  multiplying  the  number  of  attendees  by  the  number  o 
class  hours.  For  external  conferences,  seminars,  etc.,  staff  mem  ers 
who  attended  these  events  were  asked  to  report  the  number  o 
hours  spent  at  each  event. 
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Verification  of  Reported  Performance  During  the 
Audit  Period 

An  audit  of  FY2002  reported  training  hours  for  Violence  in  the 
Workplace  training  found  that  the  number  of  training  hours  was 
correctly  reported  for  all  quarters  except  for  the  fourth.  For  the 
fourth  quarter  of  FY2002,  there  were  261  training  hours  reported 
in  the  Service  Standards  Report.  Data  collected  by  the  Training 
Manager  showed  that  345  training  hours  were  administered  during 
the  fourth  quarter  of  FY2002. 

An  audit  of  FY2002  third  and  fourth  quarter  reported  training  hours 
for  Drug  and  Alcohol  training  verified  that  supervisor  Drug  and 
Alcohol  training  hours  were  reported  correctly  for  both  quarters, 
and  employee  Drug  and  Alcohol  training  was  reported  correctly  for 
the  fourth  quarter.  For  the  third  quarter,  a total  of  264  employee 
Drug  and  Alcohol  training  hours  were  reported.  However,  according 
to  the  Training  Supervisor.  3 1 6 hours  of  employee  Drug  and  Alcohol 
training  were  administered  during  the  third  quarter  of  FY2002. 

Class  logs  were  provided  for  Maintenance  Training  for  FY200I  and 
FY2002.  however  not  enough  information  was  provided  to  derive 
the  reported  number  of  hours  of  Maintenance  Training  included 
in  the  Service  Standards  Reports.  For  other  types  of  training 
hours,  records  were  incomplete  or  nonexistent  and  the  reported 
performance  could  not  be  verified. 

Problems  or  Issues  Identified  During  the  Audit  Period 

No  significant  issues  or  problems  were  identified.  However,  some 
type  of  centralized  database  or  record  keeping  system  that  could 
be  used  to  keep  track  of  all  training  hours  would  facilitate  future 
reporting  and  audits. 
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Changes  In  Measurement  Since  the  Audit  Period 

Subsequent  to  the  audit  period,  Muni  has  made  no  changes  in 
the  definition  of  measurement,  method  of  measurement,  nor 
milestones. 

Auditor  Recommendations 

All  employee  training  hours  should  bo  tracked  In  some  type  of 
centralized  system  in  order  to  facilitate  future  reports  and  audits. 
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Conclusion 

Table  F-l  summarizes  recommendations  for  each  service  standard. 


Table  F-l:  Summary  of  Auditor  Recommendations 


1 SArulnH  StflnHflrrf 

Recommendations  | 

A.  SYSTEM  RrUABILITY 

In.  % nf  uohiclos  Hint  run  on  timn  nccorilino  to 
piihlistiml  sdmdiilos. 

•Roduco  incidence  of  earlyfsharp  operations.  Schedule  evaluationiremediation  to  improve  on-time 
porformance.  Roiito  evaluation  to  improve  on-time  performance.-Targeted  improvements  directed  at 
specific,  undef-performing  lines.  Reallocation  of  terminal  checking  to  increased  NTD  checking  (after 
improvement  of  oarly/sharp  operation  in  achieved).  Measure  schedule  adherence  only  for  those  transit 
services  with  headway  intervals  of  more  than  10  min.-Coordinate  la  goal  with  other  goals  and  objectives,  ; 
parlicidarly  those  for  2a,  Ga  and  Gd.  Consider  using  cumulative,  annual  performance  to  measure  annual 
achiDVOinont  of  goals. 

2i\.  % of  schmlnlml  snrvicn  hours  tiint  nro  dnlivorod 
Olid  % of  sctiodiilod  vohiclos  that  t)o[]tn  sorvicn  nt 
tho  schodtilod  linio. 

■Dnvolop  a plan  for  maximizing  service  provision  during  "low  availability  periods."-Consider  using  cumulative.  | 
annual  porformance  to  measure  annual  achievement  of  goals. 

3n.  Missnd  soivicn  duo  to  nithor  insufficiont  vohiclos 
or  driver  unnviiihihility  ns  n |)orcaiitii[|o  of 
schodtilod  soivico  hoiiis. 

•Eliminate  this  service  standard  since  it  is  simply  an  inverse  of  2a. 

do.  % of  vohiclos  thiit  puss  puhlishod  tiino  paints 
diiiiiid  monsuiomont  poriods  iinnhlo  to  pick 
up  pnssoniiois  dun  to  ciowdinn  without  hoinp 
foltowod  within  3 miiuitos  or  loss  hy  miothor 
vohicio  on  tho  snnm  louto  with  spneo  for  nil 
wnitinii  possonpors. 

•Includo  a systemwide  pass  iip  table  by  quarter  with  both  number  and  %;  show  previous  quarters  and 
present  an  annual  summary. -Track  individual  pass  up  line  performance  over  each  quarter  of  each  fiscal 
year. -Coordinate  with  5a  so  that  all  lines  identified  as  exceeding  85%  capacity  during  peak  periods  are  i 

monitored  as  part  of  4a.  -Additional  data  available  through  the  Passenger  Service  Reports  database  should  i 
bo  used  to  conduct  follow  up  traffic  checks  on  lines  chronically  identified  as  having  pass-ups  or  peak  period  , 
loads  exceeding  85%  of  capacity. 

fin.  Took  poiiod  pnssonnoi  load  fnctnis. 

-For  noncompliant  lines,  sampling  frequency  should  be  increased  until  performance  improves. -May  be  more 
meaningful  to  capture  individual  line  load  factors  by  15  min  periods  rather  than  a for  a full  hour. -Present 
both  a quarterly  and  an  annual  performance  summary.-Coordinate  with  4a  so  that  all  lines  identified  as 
exceeding  85%  capacity  during  peak  periods  are  monitored  as  part  of  4a.  -Additional  data  available  through 
the  Passenger  Service  Reports  database  should  be  used  to  conduct  follow-up  traffic  checks  on  lines 
chronically  identified  as  having  pass-ups  or  peak  period  loads  exceeding  85%  of  capacity. 
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6a.  Actual  headways  against  scheduled  headways 
on  all  radial,  express,  cross  town,  secondary  and 
feeder  lines  for  peak,  base,  evening  and  late  night 
services. 

Measure  headway  adherence  only  for  those  transit  services  with  headway  intervals  ol  10  min  or 
less.-Schedule  evaluationiremediation  to  improve  headway  adhoronco.  Routo  evaluation  to  improve 
headway  adherence.  Reallocation  of  terminal  checking  to  mcroasod  NTD  chocking  (aftor  improvomont  of 
early/sharp  operation  in  achievodl.-Coordinate  Ga  goal  with  other  goals  and  objectivos,  particularly  those  lot 
la,  2a  and  6d.-Consider  using  cumulative,  annual  porlormance  to  measure  annual  achiovomont  of  noals 

7a.  % vehicle  availability  and  reliability  (mean 
distance  between  failure)  by  mode. 

•Consider  using  cumulative,  annual  porlormance  to  measure  annual  achiovomont  of  goals.'Consiilor  repotting 
information  on  spare  ratios  to  determine  underlying  causes  ol  vohiclo  availability  probloms.  Cortoct 
definition  ol  measurement  to  remove  the  statomont,  ‘An  additional  moasutomonl  is  added  which  shows  the 
number  of  days  when  a facility  (mode)  does  not  have  100%  availability,"  or  includo  this  information  in  tliu 
Service  Standards  Report. 

8a.  Unscheduled  absences  by  operator,  mechanical 
and  administrative  personnel. 

•Consider  establishing  a more  reasonable  goal  lor  the  % of  unschedulod  absoncos  in  the  Ailininistrntion 
Division. 

9a.  Miles  between  roadcalls  by  mode. 

‘Calculate  quarterly  MOBF  in  a consistent  manner. ‘Consider  using  cumulative,  annual  iiorlormnncu  to 
measure  annual  achievement  of  goals. 

lB.  system  performance 

. 

lb.  Passengers  carried  by  mode. 

•Remove  Data  will  be  reported  to  the  board  on  a quarterly  basis*'  from  the  Definition  of 
Measurement.‘Realislic  goals  lor  ridership  should  bo  established  based  on  economic  conditions  ns  well  tis 
available  services. 

2b.  Fare  revenues  generated  by  mode. 

•Calculate  average  fare  per  passenger. -Change  milestone  to:  "Fare  revenue  should  increase  by  1 .5%  (lor 
year  compared  with  prior  year  performance.” 

3b.  Hours  and  miles  operated  by  mode. 

Miles  and  hours  of  service  are  a direct  reflection  of  available  budget;  consider  oliminaiing  this  service 
standard  (data  will  still  be  reported  for  NTD  purposes). 

4b.  Expenses  incurred  by  mode. 

•Revise  the  performance  measurement  system  to  track  fully  allocated  costs  per  hour  of  service  by  mode. 

C.  STAFFING  PERFORMANCE 

1c.  Net  vacancies  by  position  (vacancies  remaining 
once  promotions  and  new  hires  have  been 
deducted  from  retirees  or  resignations)  for  each 
division. 

‘Limit  measurement  to  Transit  Operators  and  Maintenance  personnel. 

Pa9cF.2 


DRAFT  Final  Report 

Proposition  E Municipal  Transportation  Quality  Review 


nwomiliciiuauwii^ 

2c.  Altrilion  rates  for  now  omployoes,  by  division  and 
level. 

,S.“  »!..  s.p.r.t.  .1...  si.  ol  s.™»  shppld  bp  cmed.  -11  p.ss*le,  ps. 

availabiD  information  to  report  on  reasons  for  voluntary  separations.^ ^ j 

D.  CUSTOMER  SERVICE 

Id.  Uevolopinenl  of  annual  markotiny  plan  idonlifyiny 
specific  proyrnms  and  projects  that  will  promote 

inrrnncnrl  nntrnfUHin 

•Sot  priorities  for  marketing  efforts.  Utilize  data  from  ridership  surveys  to  evaluate  success. 

2d.  Publication  and  distribution  to  the  public  of 
schedules  for  all  trips  taken  by  all  vehicles  that 
shall  consist  of  specific  arrival  times  at  terminals 
and  ostahlished  intermediate  points. 

7ully  uiilize  capabilities  of  new  Trapeze  software  to  simplify  and  streamline  publishing  of  timetables.  | 

Ensure  that  staff  have  adequate  training  in  use  of  the  new  software. 

3d.  Operator  conduct  complaints. 

■^iarify  what  performance  is  being  measured.-Set  realistic  goals  for  reducing  operator  conduct 
rnmnlaints.Tmnrove  the  efficiency  and  accuracy  of  complaint  information  uptake. 

dd.  Annual  pnssenyer  surveys  and  follow  up  by 
manayoment. 

■In  order  to  use  survey  results  to  implement  improvements,  it  will  be  necessary  to  turttier  aeveiop  me 
connection  between  information,  analyses,  and  programmatic  action.-Amend  Definition  of  Measurement  to 
remove  focus  yroups. ^ 

Gd.  Improvements  in  public  information  royardiny 
vehicle  delays  dtiiiny  opmations  ns  well  as 
any  ynnnral  usor  infoimntion  royardiny  system 
modifications,  louto  chanyos  and  schodulos. 

•Furtbor  revise  the  Goal,  Purpose,  Definition  and  Method  of  Measurement,  and  Milestones  tor  this  service 
Standard. 

i 
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Service  Stenderd 

RKommindationi  1 

6d.  Efforts  to  improve  driver  training,  technical  as 
well  as  accident  follow  up. 

•Do  not  double  count  Violence  in  the  Workplace  training. ’Ensure  that  Definition  of  Monsmomont  nccurnloly 
describes  the  accidents  being  measured. ’Consider  reporting  the  accident  rate  in  n manner  consistent  with 
industry  standards  (e.g.,  accidents  per  100,000  revenue  milesl.'lmplement  a comprehensive  approach  ami 
coordinated  effort  to  reduce  accidents  across  divisions  and  departments  by  locusing  on  enforcement  ol 
safety  rules  and  procedures  and  thorough  review  of  alt  accidents. 

7d.  Number  of  crime  incidents  on  Municipal  Railway 
vehicles  or  in  Municipal  Railway  facilities. 

’Include  fare  evasions  issued  by  Muni  fare  inspectors.  Develop  a consistent  method  lor  coding  crime 
incidents. ’Amend  indicator  purpose  to  “To  measure  the  crime  rate  on  transit  vehicles  and  facilities. " 

E.  EMPLOYEE  SATISFACTION 

1e.  Number  of  grievances. 

’Grievances  are  not  tracked  by  division.  Either  amend  the  Definition  of  Moasuromunt  or  change 
reporting.  Establish  a goal  or  benchmark  for  the  number  of  grievances  filed. 

2e.  Speed  of  resolution  of  grievances. 

’Closely  examine  method  for  reporting  speed  of  resolution  of  non  operator  grievances  to  onsiiru  that  data 
are  reported  correctly  in  the  future. 

3e.  Longevity  of  employment. 

’The  Service  Standards  Reports  show  “Average  Years  of  Mum  Service. “ This  terminology  is  incorrect  since 
the  report  shows  average  years  of  service  with  the  City  of  San  Francisco  tor  all  job  entogorius  that  are 
not  Muni-specific. ’This  measure  does  not  provide  any  useful  information  on  employee  satisfaction.  Either 
eliminate  this  service  standard,  or  move  it  to  the  Staffing  Porforrnanco  section. 

4e.  Employee  recognition. 

-Report  on  the  actual  number  and  type  of  awards  made  quarterly  and  aimually. 

5e.  Employee  education  and  training  opportunities. 

’Track  all  employee  training  hours  in  some  typo  of  contrali/od  system  in  order  to  facilitate  future  reports 
and  audits. 
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Service  Standard  Number 

Ptrfonnanca  Indicator  Nama 

Rospunsibluindividuil  (t) 

NtlionlNygiBrd  I 
Team  Auditor  1 

a.  SYSTEM  RELIABILITY 

la.  SYSTEM  RELIABILITY 

% of  Vehicles  that  run  on  time  according  to  published  schedules  (no 
more  than  4 minutes  late  or  1 minute  early)  measured  at  terminals  and 
established  intermediate  points. 

Susan  Chotono 
SCHEDULES 

Angulo  Figuno 

2a.  SYSTEM  RELIABILITY 

% of  Scheduled  service  hours  that  are  delivered  and  percentage  of 
scheduled  vehicles  that  begin  service  at  the  scheduled  time. 

Peter  Oer 
OPERATIONS 
For  FY04  Ed  Wong 

Angulo  Figuno 

3a.  SYSTEM  RELIABILITY 

Missed  service  due  to  either  insufficient  vehicles  or  driver  unavailability  os 
a percentage  of  scheduled  service  hours 

Peter  Dor 
OPERATIONS 
For  FY04  Ed  Wong 

Angulo  Figuno 

4a.  SYSTEM  RELIABILITY 

% of  Pass-ups  determined  when  supervisory  personnel  review  all  the  linos 
checked  in  the  previous  quarter,  and  identify  the  five  lines  with  the  highest 
load  factors,  and  the  time  period  these  load  factors  occurred. 

Susan  CholoMO 
SCHEDULES 

Angulo  Figonu 

5a.  SYSTEM  RELIABILITY 

Peak  period  passenger  load  factors 

Susan  Cholono 
SCHEDULES 

Angnlo  F igonu 

6a.  SYSTEM  RELIABILITY 

Actual  headways  against  scheduled  headways  on  all  radial  express,  cross- 
town,  secondary,  and  feeder  lines  for  peak.  base,  evening,  and  late  night 
services. 

Susan  Chotono 
SCHEDULES 

Angulo  Figonu 

7a.  SYSTEM  RELIABILITY 

% vehicle  availability  by  mode. 

Deborah  Denison 
OPERATIONS  AOMIN 

Knrn  Vulclch 

8a.  SYSTEM  RELIABILITY 

Unscheduled  absences  by  operator,  mechanical  and  administrative 
personnel. 

Deborah  Denison 
Carolyn  Low  IT 

Knrn  Viilcich 

9a.  SYSTEM  RELIABILITY 

Miles  between  roadcalls  by  mode 

Jack  Hioda  LRV 
Virgil  Dennis-Motor  Coach 
Woon  WongPresidio  Trolley 
Tony  Chan  - Potroro  Trolley 
Patrick  Ho  - Cable  Car 

Knrn  Viiiclch 
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NelsonINvaaard  1 

1 

Responsible  Individual  (s) 

Team  Auditor  | 

1 Service  Standard  Number 

renermanco  iiiiiiuoiui  hpimw  — 

• i. 

111.  SYSTEM  PERFORMANCE  * 

Pnssonijors  cnrriod  by  motio  FTA  submittal  audited 

David  Makotsky 
Schedules 
Susan  Chelone 

Kara  Vuicich 

1 

21).  SYSTEM  PERFORMANCE 

Faro  rovoniios  (jonoratod  by  modo?  FTA  submittal  ■ audited 

Eloida  Leonardo 
Finance 

Bonnie  Nelson 

1 

31).  SYSTEM  PERFORMANCE  ' 

Hours  and  Milos  oporntod  by  mode  FTA  submittal  audited 

Devtd-Makefsky 
Schedules 
Susan  Chelone 

Kara  Vuicich  , 

Ah.  SYSTEM  PrUFOHMANCI: 

Exponsos  incurred  by  modo 

Fred  Clarke 
Finance 

Bonnie  Nelson 

. 

n.  STAFFING  PERFORMANCE 

Ic.  STArriNG  PFIIFOHMANCE 

Not  vaenneios  by  position  (vacancios  remaining  once  promotions  and  new 
biros  bavo  boon  doductod  from  rotiroos  or  resignations)  for  each  division. 

Lynn  Prince 
HR 

Kara  Vuicich 

2(:.STAITING  l>rHI()HMANCF 

Attrition  ratos  for  now  omployoos,  by  division  and  level 

Lynn  Prince 
HR 

Kara  Vuicich  i 

(1.  CUSTOMER  SERVICE 

III.  CDSlGMEUSFUVICf: 

Dovolopmonl  of  an  nnmial  markoting  plan  identifying  specific  programs 
and  projects  that  will  promote  increased  patronage. 

Mark  Caposino 
Marketing 

Carmen  Clark 

2(1.  CUSTGMFU  SEUVICF 

Publication  and  distribution  to  ibo  public  of  schedules  for  all  trips  taken  by 
all  vohicles  which  shall  consist  of  specific  arrival  times  at  terminals  and 
established  intermediate  points. 

Alan  Siegel 
Community  Affairs 
Bob  Caiwell 
Planning 

Carmen  Clark 

3(1.  CUSTOMCn  SERVICE 

Operator  conduct  complaints  and  their  resolution,  by  complaint,  consistent 
with  duo  process  and  required  confidontiality. 

Maria  Williams 
PSR  Unit 

Carmen  Clark 

4(1.  CUSTOMER  SERVICE 

Annua)  passenger  surveys  and  follow  up  by  management. 

Mark  Caposino 
Marketing 

Carmen  Clark 

5(1.  CUSTOMER  SERVICE 

Improvements  in  public  information  regarding  vehicle  delays  during 
operations  as  well  as  general  user  information  regarding  system 
modifications,  route  changes,  and  schedules. 

Jeff  Hamilton 
Community  and  Public 
Relations 

Bonnie  Nelson 

1 
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Service  Standard  Number 

PirfomMnci  Indicator  Nama 

Rosponsiblo  Individual  it) 

NiltonlNyoaird  I 
Turn  Auditor  1 

6d.  CUSTOMER  SERVICE 

Efforts  to  improve  driver  training,  technical  as  well  as  accident  follow  up, 
• Ron  Hundenski  provided  most  of  the  accident  data  in  the  audit 
time  period.  Now  provided  by  Paul  Lovotto 
•*  Bud  McNaughton  retired.  Currently  provided  by  Ooris  Lnnior 

Ron  Hundenski 
Paul  lovotto 
•Accidents 
Bud  McNaughton 
"Tfoining 
Miko  Kirchanski 
Doris  Lnnior 
Paul  Poturson 

Korn  Vtiicich 

7d.  CUSTOMER  SERVICE 

Number  of  crime  incidents  on  Municipal  Railway  vehicles  or  in  Municipal 
Railway  facilities. 

Michollo  Blalock 
Socurity 

Knrn  Viiiclch 

e.  EMPLOYEE  SATISFACTION 

^ _ 

le.  EMPLOYEE  SATISFACTION 

Number  of  grievances 

Miku  Cosoy 
Human  Rosourcos 
Mnhol  Shaw 
fluinmi  Rosourcos 

Kara  Viiiclch 

2e.  EMPLOYEE  SATISFACTION 

Speed  of  resolution  of  grievances 

Miko  Cosoy 
Human  Rosourcos 

Kara  Viiiclch 

3e.  EMPLOYEE  SATISFACTION 

Longevity  of  employment 

Michael  lum 
Human  Resourcns 
Hannoh  Slorosolsky 
Human  Rosourcos 

Kara  Viiiclch 

4e.  EMPLOYEE  SATISFACTION 

Employee  Recognition 

Emptoyoo  Rocognition 
Program  Orgoni/or 
Orango  Roso 

Knrn  Viiiclch 

5e.  EMPLOYEE  SATISFACTION 

Employee  education  and  training  programs 

Ooborah  Ooniion 
Vornon  Crowloy 
Rich  Dale 
Patty  Deloon 
Barbara  Conway 
Paul  Potorson 

Knrn  Vuicich 
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